Fastcar: Cache Attentive Replay for Fast Auto-Regressive Video Generation on the Edge
Authors: Xuan Shen
Weize Ma
Yufa Zhou
Enhao Tang
Yanyue Xie
Zhengang Li
Yifan Gong
Quanyi Wang
Henghui Ding
Yiwei Wang
Pu Zhao
Jun Lin
Jiuxiang Gu
Venue: ICLR 2026 Poster (2026)
Keywords: Video Generation
Efficient Video Generation
Auto-Regressive Video Generation
Primary area: generative models

Abstract: Auto-regressive (AR) models, initially successful in language generation, have recently shown promise in visual generation tasks due to their superior sampling efficiency.
Unlike image generation, 
video generation requires a substantially larger number of tokens to produce coherent temporal frames, resulting in significant overhead during  decoding. 
We first make specific key observations: (i) MLP modules in the decode phase dominate the inference latency, and (ii) there exists high temporal redundancy in MLP outputs of adjacent frames. 
With the insights, we propose **FastCar** to accelerate the decode phase for the AR video generation  by exploring the temporal redundancy. The Temporal Attention Score (TAS) is proposed to determine whether to apply the replay strategy (i.e., reusing cached MLP outputs from the previous frame to reduce redundant computations)  with detailed theoretical analysis and justification.
Furthermore, we develop a hardware accelerator on FPGA with Dynamic Resource Scheduling  based on TAS to enable better resource utilization and faster inference.
Experimental results demonstrate the effectiveness of our method, which outperforms traditional sparse attention approaches with more than 2.1x decoding speedup and higher energy efficiency on the edge.
Furthermore, by combining FastCar and sparse attention, FastCar can boost the performance of sparse attention with alleviated drifting, demonstrating our unique advantages for high-resolution and long-duration video generation.
Code: https://github.com/shawnricecake/fast-car

Published as a conference paper at ICLR 2026
FASTCAR:
CACHE ATTENTIVE REPLAY FOR FAST VIDEO GENERATION ON THE EDGE
Xuan Shen Weize M Yufa Zhou² Enhao Tang Yanyue Xie Zhengang Li Yifan Gong"
Quanyi Wang' Henghui Ding Yiwei Wang" Pu Zhao't Jun Lin Jiuxiang Gu "Northeastern University "Nanjing University DDuke University 4Adobe NUIST Fudan University 'University of California, Merced shen. xu@northeasterm.eem jlin@nju ju. edu. cn zhao. whw.. www.ortheaseeer..cco.om edu jiquBadobe.com
weizemann weizema@smail, www.l.lnnnnim..ma...... www..ddu.cuun edu.
ABSTRACT
Auto-regressive (AR) models, initially successful in language generation, have recently shown promise in visual generation tasks due their superior sampling efficiency. Unlike image generation, video generation requires substantially larger number tokens to produce coherent temporal frames, resulting in significant overhead during decoding. We first make specific key observations: (i) MLP modules in the decode phase dominate the inference latency, and (ii) there exists high temporal redundancy in MLP outputs of adjacent frames. With the insights, we propose FastCar to accelerate the decode phase for the AR video generation by exploring the temporal redundancy. The Temporal Attention Score (TAS) is proposed to determine whether to apply the replay strategy (i.e., reusing cached MLP outputs from the previous frame to reduce redundant computations) with detailed theoretical analysis and justification. Furthermore, we develop hardware accelerator on FPGA with Dynamic Resource Scheduling based on TAS enable better resource utilization and faster inference. Experimental results demonstrate the effectiveness of our method, which outperforms traditional sparse attention approaches with more than 2.1 x decoding speedup and higher energy efficiency on the edge. Furthermore, by combining FastCar and sparse attention, FastCar can boost the performance of sparse attention with alleviated drifting, demonstrating our unique advantages for high-resolution and long-duration video generation. Code: https:://////t / thub. com/ haam/shawwng shawnr cecake/ fast-car
INTRODUCTION
Recently, there has been growing interest in extending the Auto- (AR) framework of Large
Language Models (LLMss) (Radford et al. [009 Touvron et al. [0023] Grattafiori et al. 2024
Zhao
al.
2024a) visual generation tasks (Sun et al. [2024] Wang et al.
2024a Han al. 2024
Tian
et al.
2024 Weng et al. 2023 Deng et al. 2024
Jiao et al. 2025
Xie et al.
2024 Sun et al.
2024
Luo al. 2024 Kondratyuk al.) 2023 Wang al.) (024b). The works
(Sun al.
2024
Tian
et al.] 2024 Sun et al. [2024] Han et al. 22224] Wang et 20224b convert images into tokens and
apply AR models to generate image tokens with next-token prediction. The generation quality is
surprisingly strong, often surpassing diffusion-basee based techniques in perceptual fidelity and semantic coherence.
As video becomes dominant medium across entertainment, communication, etc., synthesizing coherent high-quality videos from minimal inputs presents a compelling research challenge (Xiong
et al. 2024 Xing et al.) 2024 Li et al. 2024a; Melnik et al. (024).. Prior works (Lin et al.) 2024a
Zheng et al. 2024; Peng et al. 2025 Hong et al.] 2022 Yang et al. 2024, Kong et al. 2024)
leverage Diffusion Transformers (DiT) Peebles & Xie 2022) to develop video generation models
with superior generation performance. at the cost of substantial computations and massive memory
demands (He al. 2025 Jin et al.) 2024 Xu et al. 2025 Lu et al.) 2025 He al.) 2024, Li
et al. 2025b). These characteristics limit their practical applications and deployments for resource-
"Equal Contribution
1 "Correppnnding Author

---

Published as a conference paper at ICLR 2026
Tokens for Previous Frame
Tokens for Current Frame
I Latency (8)
Profill
Decoding
4
256
3000
KV Cache
KV Cache
2500
2000
1500
Whole Attn
Whole Attn
1000
Module
Module
Seq. Length 2k
4k
Fowwrorward
Feedforward
256
(MLP) Module
Average TAS
(MLP) Module
score 2 r?
Traffic Light
Replay
Cache
Replay
2k
4k
6k
8k
Attn Linear
QKY
MLP
Figure Left: FastCar framework. We replay the cache from the previous frame skip the computations for MLP in decoding. Replay is triggered when the average TAS exceeds predefined Latency cost of both prefill and decode phases for different sequence lengths. Right Bottom: Detailed latency cost of the decode phase for different sequence lengths.
threshold 7. Right Top:
constrained environments (Luu ct al
2025
Zhu et al. 2023 Li ct al. 2023b; Shen et al. 2025c Yang
et al. 2023a Zhan al., 2024b Shen et al. 2025b Yang et al. 2025 such mobile devices
Field Progra-mooaaa Gate Array (FPGA) with tight constraints for energy efficiency, memory, etc.
Motivated by the scalability and fast decoding capabilities of AR transformer-baee frameworks in generative tasks, an increasing number of works (Wu et al. 2024 Deng et al. 2024 Weng et al.
2023
Xie al. 2024 Kondratyuk al. 2023) have adopted AR frameworks for video generation
tasks. To further improve its efficiency, model compression strategies (such as model pruning and
quantization (Zhang et al. [2022 Lin et al. 2024bb Ma al.| 2023) Shen et al.) 2024a 2025c Li
et al. spatial redundancy optimizations (such as sparse attention (Xiao al. 2023b Rehg 2024 Hooper
2024b Zhao et al. 2024b Zhan et al. 2024c Xiao et al. 2023a] Liang et al.) 2025) and
et al.
2024
Liu et al.
2024 Ge et al. 2024 Li et al.)
2024c. Shen et al. 2025a Li et al. (025a)
and e efficient sampling methods (Spector Re 2023 Yang al. al.| 2023b Miao et al. 2023 Ning
et al. 2024 Teng et al.
2024 Shen et al. 2025b
He et al. 2025
Kong et al. 2025 Shen et al. 2023
2024b 2025d)) are investigated. However, the inherent temporal redundancy specifically introduced
by videos with multiple sequential frames remains largely unexplored in AR video generation.
Specific Deep Insights. To explore the redundancy for superior efficiency, we first perform a detailed latency profiling and similarity analysis between different frames. As shown in the right of Figure [ we identify that the MLP modules (rather than the attention modules) in the decode phase dominate the inference latency. Meanwhile, according to Figure 2 the outputs of adjacent frames for the same MLP module exhibit relatively high resemblance/simiiartyy indicating high temporal redundancy.
Framework with Theoretical Justification. Based the deep insights specific for AR video generation, we propose FastCar for efficiency optimization. The Temporal Attention Score (TAS) is proposed determine whether skip the computations of the MLP modules (Figure DT. If skipped, the cached outputs from the previous frame are directly reused as current outputs (similar to video replay) due their high similarity. Skipping computation-intensin MLP modules leads to substantial accelerations. We further provide detailed theoretical analysis to formally characterize how our TAS controls the output differences across adjacent frames, thereby justifying the design of FastCar.
Hardware Accelerator. A flexible and ceffciencc--rrience hardware accelerator is further developped to support kernel fusion and custom instruction programmability. thus allowing direct reuse of cached outputs and enabling conditional execution of MLP modules. Specifically, to handle varying workload sparsity, we propose Dynamic Resource Scheduling (DRS), which leverages attentivity to dynamically allocate computational resources. DRS, integrated into lightweight control logic, helps alleviate bandwidth pressure and improves overall resource efficiency, thereby enabling faster inference.
Comprehensive Experiments. Experimental results show that FastCar not only surpasses sparse attention (SA) methods with better generation quality, but also achieves faster decoding with improved energy efficiency on FPGA. Additionally, FastCar complements SA approaches by mitigating their
2

---

Published as a conference paper at ICLR 2026
drifting issues. By combining FastCar and SA, FastCar significantly boosts the generation quality of SA with faster inference and better long-range temporal coherence.
Our contributions are summarized follows,
1. We perform the latency profiling and similarity analysis between different frames to explore the temporal redundancy in MLP modules. We then propose FastCar framework to accelerate AR video generation by replaying MLP modules using cached outputs from the previous frame.
2. Our theoretical analysis demonstrates that the similarity of MLP outputs across adjacent frames correlates with the attentivity, and this correlation is consistent across various model depths, thus justifying the design of FastCar with TAS (i.e., the atlentivity) to guide replay decisions.
3. We develop an efficiency-orientee hardware accelerator with DRS, enabling dynamic allocation of computational resources to enhance resource utilization and accelerate inference on FPGA.
4. Experimental results show that FastCar outperforms SA methods in generation quality by alleviating drifting issues of SA, and achieves more than 2. x speedup. thereby enhancing scalability and efficiency for high-resolution and long-duration AR video generation.
RELATED WORK
Auto- Sego-Reeressive Visual Generation. Prior works (Sun al. 2024 Wang et al. 2024a, Han et 2024, Tian et al. 2024 Jiao et al. 2025: Xie et al. 2024 Sun et al., 2024 Luo et al. (2024) apply the AR framework for image generation. demonstrating its potential to outperform diffusion- based models. In particular, VAR (Tian et al., (024) introduces next-scale prediction progressively generate token sequences across multiple resolutions, demonstrating the effectiveness AR methods with enhanced image quality. Inspired by this, several works (Wu al. 2024 Deng et al. [024 Weng et al. 2023 Kondratyuk et al., 2023) adopt the AR framework for video generation models.
Efficient Techniques for Auto-Regressive Visual Generation. AR image generation models (Sun) et al., 2024 Wang et al., 2024a), typically require n² sequential forward passes to generate an image represented by n X tokens, resulting in significant inefficiency, which is further exacerbated when extending to video generation (Wu et al.) (2024) with multiple image frames. Some works (He et al.) 2024| Teng et al., 2024) accelerate the sampling process decode phase, utilizing contextual cues from neighboring tokens to reduce redundant computations.
3 DEEP INSIGHTS FOR AUTO-REGRESSIVE VIDEO GENERATION
To effectively accelerate AR video generation, first perform detailed profiling for the inference latency and computations of VILA-U (Wu et al. 2024). We then provide the following specific deep insights: (i) The decode phase takes significantly longer than the prefill phase. (ii) The MLP modules dominate the latency of the decode phase. (iii) The outputs of an MLP module exhibit great similarity those of its previous frame. Next we demonstrate our detailed observations and analysis.
Prefill Phase v.s. Decode Phase. We compare the latency of the prefill phase and decode phase under different input sequence lengths from 2k to 8k, during AR video generation. As shown in the right top of Figure [ the decode phase takes significantly longer than the prefill phase under various input lengths, as it needs to generate a large number of visual tokens for videos with multiple frames.
Attention Modules v.s. MLP Modules. We further explore detailed latency profiling for different decoding modules. The bottom right of Figure I shows that under varying input sequence lengths, MI.P modules consistently dominate the overall latency. This observation underscores the distinct computational characteristics of AR generation compared with diffusion-basee methods. Specifically, in diffusion transformers, all visual tokens processed simultancously through iterative denoising, with attention modules as the primary computational bottleneck. In contrast, AR models generate tokens sequentially, where attention modules only contribute marginally the overall latency. As result, efficiency-orienen techniques designed for attention modules are less effective in AR.
Spatial Redundancy Temporal Redundancy. Spatial Redundancy is commonly explored in image generation with SA mechanisms (to reduce computations) or efficient sampling (to generate fewer tokens). In contrast, temporal redundancy in video generation remains largely overlooked, as
3

---

Published as a conference paper at ICLR 2026
Cosine Similarity
Similarity Frame Before
Similarity Frame Refore
Similarity Frame Before
0.8
0.6
0.4
0.2
Layer
2
3
7
8
10
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
Figure 2: Cosine similarity for MLP outputs between neighboring frames for all 32 MLP modules.
prior works focus on image generation. To explore temporal redundancy, we present cosine similarity between outputs of an MLP module and those of its neighboring frames in Figure 2 The MLP outputs exhibit high similarity with their most recent frame, demonstrating high temporal redundancy.
Motivation. Based on the observation that MLP modules dominate the overall latency, we mainly optimize the computations of MLP modules for acceleration. Temporal redundancy with high similar- ities between the MIP outputs of neighboring frames further motivate us to cache the corresponding hidden states from the current time step for reuse its next time step, thus avoiding its computations.
FASTCAR FRAMEWORK DESIGN
We demonstrate FastCar framework in this section. In general, when the proposed Temporal Attention Score (TAS) indicates high similarity, we then directly reuse the cached outputs from its previous frame as the outputs the current frame, thus skipping the MLP computations.
We first provide specific definitions for AR video generation. Then we demonstrate FastCar in great details. Theoretical analysis are further provided for the rationality and justification of FastCar.
4.1 AUTO-REGRESSIN VIDEO GENERATION
We model a video V as a temporal sequence T' frames with visual tokens in each frame. With Vvis denoting a finite vocabulary of visual tokens, it can be formulated as follows,
V == a t= 1,..., T;
= 1i.....,,NN N}, 2,, Viss
(1)
Flattening the temporal-spatial grid yields a sequence of length = T N. We denote the flattened token index as j = (t, == (I- i, where denotes the frame index and denotes the index of the spatial position. Since frames are consecutively ordered, it satisfies: (t-1, = (t, - N.
For transformer layers, we define the hidden states: X E Raxd where d denotes the hidden size.
We use batch size of B = for simplicity, with all results readily extendable to B > through broadcasting. The objective of AR video generation is to model the joint distribution:
P(V) = II P(z, | xEs))
(2)
4.2 KEY MODULES
We now formalize the key modules in AR video generation and our FastCar framework. The model has multiple blocks, with an attention module and an MI.P module for each block, as defined below. Definition 4.1 (Attention Module). Given hidden states E Baxd attention output is computed as:
Attn (X) Softmax
QK Vd
)
VERRX with Q XWo, == XWw, V == XWv.
(3)
where Wo. WK. Wy E dxd are the query, key, and value projection matrices. Definition 4.2 (MLP Module). Given input hidden states X E Raxd the MLP module is defined as: MLP(X) = (act(XWe) (XWo)) WD E Raxd
(4)
where act(-) a non-linear activation functiooo (e.g., SiLU), o denotes element-wise multiplication, and Wa, WG E Rd Wp Rd4 learned parameters with the intermediate size de-
4

---

Published as a conference paper at ICLR 2026
Next define TAS measure the token similarity of adjacent frames and guide replay decisions. Definition 4.3 (Temporal Attention Score). The temporal attention score spatial position i and t-th frame is defined as the scaled dot-product between the query vector qj of token j = (1,3) and the
key vector k of its aligned token j (t-1,
(gs K
81,5
E R.
(5)
In our FastCar framework, due to causal decoding, TAS is obtained directly from the attention module preceding the MLP, incurring no additional computational cost.
4.3 CACHE ATTENTIVE REPLAY FOR FAST GENERATION (FASTCAR)
In FastCar, with TAS, we enable attentive replay in MLP modules by manually setting threshold to filter tokens of adjacent frames with higher attentivity. When TAS is larger than r, which indicates significant temporal similarity, we skip MLP computations by reusing the outputs of its previous frame at the same spatial location, as illustrated in the left of Figure [
Specifically, for each transformer block, at frame t 1, for each spatial token index i, we cache the MLP output as follows:
1,,,, = MLP (X) + X) (1-,)),;
(6)
(m) 1 ( m= between token (t, i) and its aligned token (I 11-1,
At frame we evaluate the of TAS across all h attention heads (Definition and compute the mean score:
(m)
(7)
h
When the mean score exceeds a predefined threshold r, i..., St,; 2 т, we then skip the following MLP computations of this specific token (t, i) and reuse the cached output for the replay in the block:
if 3 27 MLP (Attn(X) + () Otherwise
Ya.9. =
(8)
Otherwise, we still perform the normal MLP computations. In the AR model, there multiple transformer blocks with the same architecture following the same computation pattern. We apply FastCar for each block. This selective replay mechanism reduces MLP computations by leveraging temporal consistency across adjacent frames.
4.4 THEORETICAL SIMILARITY ANALYSIS
We now formally characterize how TAS controls the output differences across adjacent frames, thereby justifying temporal replay based on TAS.
We first relate temporal attention similarity to the difference attention outputs.
Theorem 4.4 (Attention Score Controls Attention Output Difference). Let X E B be the hidden
states, where each row Tj Rdd represents the hidden state of token j. Let Attn(X) denote the attention output defined in Desinition 4.1 For tokens j = (1, and j== (1-1, aligned at the
same spatial position, define the temporal attention score 3t as in Definition Assume that:
(1) The hidden states are bounded: ||æz|12 S M for all j;
(2) The projection matrices satisfy [[all2, |W l2 A;
(3) The query and key vectors are normalized: ||æحl| || ll2 for all j.
Letry: I[Q - Wx l| denote the projection difference.
Then, under the Lipschitz continuity of the attention, there exists a constant C such that:
[Attn(X)): -Attn(X)--llee Sll2 S (V1 $1,1 + +M)
(9)
Thus, a larger 81, implies smaller atlention output difference up model-dependent offset M.
5

---

Published as a conference paper at ICLR 2026
Instruction
Instuction
Mapping Register
Instruction
Feich
Register
Conprol
Danch Cure 0
Batch 0 Core
Care
DRS
ID
Basch Core
Control
Replay
Banch - Core 3
DRS
Batch Core
DMA
MU
AXI
255
Betch 4 - Core 0
Average TAS
Rarch S. Care
Batch 5 Core
Duta
Traffic Light
SRAM
Figure 3: Left: The top-level block diagram of our hardware accelerator. Right:
The DRS diagram.
Transformers often apply normalization techniques (such as LayerNorm or explicit vector normaliza- tion) to control query and key magnitudes. Thus assuming ||رر||2 ||k; lll is reasonable and standard for theoretical analysis Shen et al. (2025b c). We delay the full proof to Appendix C
Next, we relate input and attention similarity to MLP output similarity.
Theorem 4.5 (Attention and Input Similarity Implies MLP Output Similarity). Let MLP(.) de- note the MLP module defined in Definition (4. Let Y,,- MLP (Attn(X) + X),, and Y MLP (Attn(X X) + x) denote the MLP outputs at tokens and j-
Assume that MLP(.) is T.-Liisshhttz continuous. Then
1 Y--,:ll S (IIX,, - -xj-,llz + Atttn(X)s. -ttn(x))-,,ll))
(10)
The proof is demonstrated in Appendix C Finally, combining the two results, we directly relate TAS to MLP output similarity.
Theorem 4.6 (Temporal Attention Score Controls MLP Output Similarity). Let X RRad be the hidden states, and let Yj.: and Yj- denote the MLP outputs at tokens and j- Let st, denote the temporal attention score. Under the assumptions of Theorem 4.4 and assuming the MLP is
L-Lipschitz, there exists a constant C > 0 such that:
1r Y, 1/ 1 1
X3 12 /1
81,1 +
(11)
The proof is demonstrated in Appendix C Theorem 4.6 formally establishes that high TAS, combined with input similarity, guarantees small MI.P output deviation across adjacent frames. This justifies the use of thresholds on TAS to dynamically skip MIP computations during decoding, enabling efficient temporal replay with controlled quality loss. Moreover, as TAS is computed layer-locally, offers stable and inppeiindeeendentt signal for runtime adaptation (Remark 4.7.
Remark 4.7. The TAS st,; depends only on the local query and key vectors at the current layer and is independent of model depth. It captures instantaneous similarity without accumulating information across layers, making a stable, efficient, and fine greengraindd signal for dynamic computation reduction during auto-regressive decoding.
HARDWARE DESIGN
We develop programmable hardware accelerator (Shen et al. (2024c): : Li et al. (2022 Wu et al. (2022); ; IZhan al. (202): [i et. al. (2023a), as shown in the left of Figure Pre-compiled instructions are loaded via the AXT bus with the Fetch module, and dispatched to the corresponding instruction FIFO OFirst-nn-First-Out), The Control module manages Matrix Unit (MU) and Vector Unit (VU) to perform matrix multiplication and vector computation, while the Direct Memory Access (DMA) module is responsible for loading data from off-chip memory (i.e., DDR or HBM). The instruction FIFO receives control signals from the Control Module to coordinate the computations of each unit. The Dynamic Resource Scheduling (DRS) module is employed to address the computational workload imbalance caused by dynamic replay from the FastCar framework.

---

Published as a conference paper at ICLR 2026
Table 1: Main results our method compared with sparse attention method StreamingLLM Xiao et al. (2023)), where dense attention is retained in the first frame for fair comparison. Local size denotes number of local tokens for sparse attention. Detailed VBench scores are Appendix A
Latency is tested for whole generation of one video. Power efficiency is computed by GFLOPs/W.
Method
Replay
Local Size
PSNR
SSIM
LPIPS
VBench Score Quality
TFLOPs
Latency
Power
Ratio
Total
Semantic
(s)
Bm..
Dense
74.1%
76.4%
65.0%
31.79
(1xx
1.47
256
18.25
51.54
33.59
72.1%
74.6%
62.5%
30.95
670.5 (1.02x)
1.51
/
128
13.14
33.61
54.34
60.7%
61.9%
55,9%
30.82
666.9 (1.03x)
1.52
Sparse
64
13.47
33.79
53.54
62.6%
63.3%
60.2%
30.76
666.3 (1.03x)
1.52
Attn.
32
13.34
33.14
53.82
61.4%
61.3%
62.0%
30.72
663.9 (1.04x)
1.52
16
13.30
32.02
53.75
64.5%
64.8%
63.3%
30.70
662.7 (1.04x)
1.53
10%
18.57
53.32
27.31
73.4%
75.5%
65,2%
30.09
629. 1 (110x)
1.61
20%
17.94
51.01
27.57
73.2%
7.53%
65.
28, 64
556.8 (1.24x)
1.82
30%
17.87
50.29
28.02
72.4%
74.3%
64.7%
27.18
525.2 (1.31
1.93
40%
17.68
50.14
28.15
71.8%
73.0%
67.2%
25.73
487.2 (1.42x)
2.08
Ours
50%
17.85
50.11
28.08
71.5%
72.7%
666.6%
24,27
(1
2.13
60%
17.85
50.55
28.72
71.4%
72.7%
66.2%
22.33
(1.53))))
2.24
70%
17.86
50.18
28.79
71.2%
72.3%
66.9%
20.88
(1.66()
2.43
80%
17.71
49.01
29.50
71.5%
73.0%
65.6%
19.42
390.7 (1.76x)
2.59
DYNAMIC RESOURCE SCHEDULING (DRS)
The FastCar framework dynamically determines whether to adopt the replay strategy to skip compu- tations for certain MLP modules based on the TAS of different batches. Due to the multi-core design, computations for different batches in dense mode are mapped to distinct cores statically. However, the dynamic FastCar introduces computational workload imbalance across different cores, as the number of MLPs computed for different batches may vary and difficult to predict during infer- ence. Morcover, we employ static compilation to pre- vre-generate scheduling instructions. Exhaustively enumerating all possible cases would incur an unaffordable large instruction storage overhead.
To address this, propose the DRS to balance the computational workloads, as shown in the right of Figure 3 After computing TAS, we configure an on-chip computation mapping table. A 32-bit Index Register, which stores the status of each batch (0-replay, 1=compute), is established to determine whether computation should skipped. 32 Mapping Registers, each with log, (num_cores) bits, determine the target core for executing which batch. We adopt a round-robin assignment strategy to ensure balanced workload distribution among the cores.
When pre-compiled instructions are loaded and the replay mechanism is triggered, the prefetched instructions are forwarded to the DRS for processing. The DRS then performs scheduling decisions by consulting the Index Register to determine whether discard instructions for replayed batches or dispatch them to the appropriate cores based on the Mapping Registers. Notably, the DRS incurs minimal overhead by completing its dispatch operations in just hundreds thousands of cycles, which is negligible compared to the thousands of cycles required for actual instruction execution.
6 EXPERIMENTAL RESULTS
6.1 EXPERIMENTAL SETUP
We adopt the AR video generation model from VILA-U (Wu et al. 2024). Videos are generated with 8 frames in 256x256 resolution, where each frame is decoded by 256 tokens. VILA-U is the only available open-source model at this time (Section 2). We evaluate quality of generated videos with VBench (Huang et al. 2024), and the similarity metries including PSNR, SSIM, and LPIPS (Zhang et al., 2018). In detail, we compute the average similarity across all frames except the first one, as our method generates the first frame in the same manner as the baseline. We generate videos using prompts from VBench with batch size of 5, fixed random seed of 42, and a classifier- free guidance of (Ho & Salimans 2022) on 100 GPUs Additionally, there are no available direct baselines in this novel area, and compare
method against the sparse attention (SA) approach StreamingLLM (Xiao et al.) 20236). We set the sink size by extending the prefill length by 256 to ensure the first frame preserved throughout video generation for fair comparison.
7

---

Published as a conference paper at ICLR 2026
LPIPS +
VBench Score
Replay Ratio
VBench Score
34
74%
100%
74%
33
Consistent
87%
Inconsistent
80%
73%
73%
31
60%
30
72%
72%
29
20%
28
27
71%
0%
71%
Ratiol0%
20%
30%
40%
50%
60%
70%
80%
f
0
-0.5
-1
-1.5
-2
-25
-3.5
-8 -16
Figure 4: Left: Ablation study comparing consistent vs. inconsistent threshold settings with respect to LPIPS and the VBench total score. Full results are provided in Table Appendix A Right: Ablation study on the effect the threshold 7 on replay ratio and VBench total score. Full results are
reported in Table 6 Appendix A
Replay Ratio
H=-1
r=-2
100%
80%
60%
40%
20%
0% Layer
2
3
6
7
9
10
11
12
13
14
15
17
18
20
21
22
23
24
25
26
27
28
29
30
31
32
Figure 5: Replay ratio distribution across layers for thresholds
- 1, -2 and 4, respectively.
For hardware implementation, we adopt the Xilinx Alveo U280 FPGA as the target platform with a chiplet design. We implement multiple accelerator cores on the FPGA ensure physical implemen- tation feasibility. Latency and power are tested using a prefill sequence length of 256.
6.2 MAIN RESULTS
We provide the main results with different replay ratios compared with the SA method in Table T The detailed VBench scores for all results in different dimensions are included in Appendix A We make the following observations: (i) The SA methods models effectively as the MIP modules dominate the computations/latenn and thus optimizing attention modules does not effectively address the bottleneck. (ii) Our method effectively reduces the computations and achieves significant accelerations with better power efficiency. With an 80% replay ratio, our method can reduce 45% computations with a 1.77x acceleration. (iii) Our method better maintains the generation quality than the SA methods. For similarity metrics including PSNR, SSIM, and LPIPS, with gradually increasing sparsity, our generation quality only degrades marginally, which significantly outperforms the SA method with notable degradations. For the VBench scores, we can make similar observations. Meanwhile, our method achieves higher power efficiency compared to the SA method. Also, we provide the visualization with our method in different replay ratios in Appendix B Visualization shows that vidco quality well preserved across different replay ratios.
not able accelerate AR video generation
6.3 ABLATION STUDY
Threshold Distributions. We conduct an ablation study on threshold distribution by applying either consistent or layer-wise varying (i.e., inconsistent) thresholds across all layers, while maintaining the same overall replay ratio. As shown in the left side of Figure 4 consistent threshold achieves better performance with lower LPIPS and higher VBench score than inconsistent thresholds, which verifies the effectiveness of Remark 4.7 More discussions can be found in Section A.2 of Appendix.
Threshold Values. Meanwhile, we ablate the threshold values to evaluate their impact on model performance, as shown in the right side of Figure 4 When < -2.5, if we continue to decrease т, the generation quality does not further degrade while higher sparsity with faster inference can be achieved, demonstrating the robustness of FastCar. Additionally, generation model achieves the highest replay ratio of 87% when T xe -8, indicating that only 1 13% of the MIP modules actually required during the generation process.
observe that the AR video
8

---

Published as a conference paper at ICLR 2026
Table 2: Additional results for the combination of the sparse attention method and our method. More results are included in Table 7 Appendix A
Replay Ratio
Local Size
VHBench
GHLOPs
Latency (6)
Power
Method
Total
Quality
Semantic
E
Dense
74.1%
76.4%
65.0%
31.79
689.7 (1x)
1.47
87%
256
17.44
47.57
31.27
71.8%
73.3%
65.7%
17.61
354.46 (1.95x)
2.85
Ours
87%
128
17.29
46.79
32.10
71.7%
73.3%
65.7%
17.49
342.25 (2.02x)
2,96
Sparse
87%
64
17:27
46.70
32.09
71.6%
73.1%
65.5%
17.43
334.57 (2.06x)
3.02
Attn.
87%
32
17.27
46.75
31.96
71.9%
73.4%
65.9%
17.40
327.36 (2.11x)
3.09
87%
16
17.27
46.49
32.37
71.6%
73.1%
65.7%
17.39
324.31 (2.13x)
3.12
Past Haseline
Sparse Altn
Combine
Figure 6: Visualization for the prompt "A dog wearing sunglasses on the beach.". The second and third rows are generated with threshold T -4 (i.e., 82% replay ratio). The third and fourth rows are generated with sink size that extends the prompt length by one frame and 64.
6.4 ADDITIONAL ANALYSIS
Replay Ratio Distribution. We visualize the replay ratio distribution across layers for different thresholds in Figure We observe that the model tends to replay at the shallow and deep layers, while replay is less likely to occur in intermediate layers. This indicates that intermediate layers play critical role in capturing temporal dynamics and contribute most significantly generation quality in auto-regressive video models.
Combination with Sparse Attention. We further provide additional results achieved by combining SA method and our method in Table 2 The detailed VBench scores for all results in different dimensions are included in Table at Appendix [A] The results show that our method can significantly boost the performance of the SA method through straightforward combination. This validates the effectiveness of our method as complementary enhancement to existing SA approaches. We further visualize the results of our method, the SA approach, and their combination in Figure 6 directly illustrate how our method alleviates drifting when integrated into sparse attention.
7 CONCLUSION
We propose FastCar, framework to accelerate AR video generation on the edge. We show that the similarity across adjacent frames correlates with attentivity and independent with model depths. We then introduce replay strategy that reuses cached MLP outputs from the previous frame to reduce computation. Furthermore, the DRS design is adopted to improve resource utilization and inference speed on FPGAs. Results show that method outperforms SA approaches and complements them with more than 2. x speedup, enabling better scalability for high-resolution, long long-duration video generation. In future work, we plan extend framework broader range of model families.

---

Published as a conference paper at ICLR 2026
REPRODUCIBILITY STATEMENT
The proposed framework developed on top of auto-regressive video generation models and leverages the reuse of MLP outputs reduce computation. The theoretical analysis supporting the similarity of MLP outputs across adjacent frames correlates with the attentivity is provided in the paper. All hardware metries, including latency and power efficiency, are empirically evaluated real devices. The full codebase and implementation details will be released publicly upon acceptance of the paper.
LLMs USAGE STATEMENT
We report that LLMs were used only for stylistic editing of the manuscript text. All scientific content, analysis, and conclusions remain the sole work of the authors.
REFERENCES
Haoge Deng, Ting Pan, Haiwen Diao, Zhengxiong Luo, Yufeng Cui. Huchuan Lu, Shiguang Shan, Yonggang Qi, and Xinlong Wang. Autoregressive video generation without vector quantization. arXiv preprint arXiv:2412. 141129 2024.
Suyu Ge, Yunan Zhang, Liyuan Liu, Minjia Zhang, Jiawei Han, and Jianfeng Gao. Model tells you what to discard: Adaptive KV cache compression for LLMs. In The Twelfth International Conference on Learning Representations, 2024. URL https: /optnee///penrevlee wwt.eewwwweeeootteeeeeeee
id=uNzPpDPMyo
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint wwXiv:240722178. 2024.
Jian Han, Jinlai Liu, Yi Jiang, Bin Yan, Yuqi Zhang, Zehuan Yuan, Bingyue Peng, and Xiaobing Liu. Infinity: Scaling bitwise autoregressive modeling for high-resolution image synthesis, 2024. URL
https://arxiv. wttp/oog/////a/oeo / 04431
Yefei He, Feng Chen, Yuanyu He, Shaoxuan He, Hong Zhou, Kaipeng Zhang, and Bohan Zhuang. Zipar: Accelcrating regr-ssgressive image generation through spatial locality. arXiv preprint arXiv:2412.04002 2024.
Yefei He, Yuanyu He, Shaoxuan He, Feng Chen, Hong Zhou, Kaipeng Zhang, and Bohan Zhuang. Neighboring autoregressive modeling for efficient visual generation. arXiv preprint 10066 2025.
Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598 2022.
Wenyi Hong. Ming Ding, Wendi Zheng. Xinghan Liu, and Jie Tang. Cogvideo: Large-scale pretraining for text-to-video generation via transformers. arXiv preprint arXiv:2205. 15866, 2022.
Coleman Hooper, Schoon Kim, Hiva Mohammadzadeh, Michael W Mahoney, Yakun Sophia Shao, Kurt Keutzer, and Amir Gholami. Kvquant: Towards 10 million context length 11m inference with kv cache quantization. arXiv preprint arXiv:2401. 180799, 2024.
Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, Yaohui Wang, Xinyuan Chen, Limin Wang. Dahua Lin, Yu Qiao, and Ziwei Liu. VBench: Comprehensive benchmark suite for video generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024.
Siyu Jiao, Gengwei Zhang, Yinlong Qian, Jiancheng Huang. Yao Zhao, Humphrey Shi, Lin Ma, Yunchao Wei, and Zequn Jie. Flexvar: Flexible visual autoregressive modeling without residual prediction. arXiv preprint arXi::5002.20313. 2025.
Yang Jin, Zhicheng Sun, Ningyuan Li, Kun Xu, Hao Jiang, Nan Zhuang, Quzhe Huang, Yang Song, Yadong Mu, and Zhouchen Lin. Pyramidal flow matching for efficient video generative modeling. arXiv preprint arXiv:2410.0595 2024.
10

---

Published as a conference paper at ICLR 2026
Dan Kondratyuk, Lijun Yu, Xiuye Gu, José Lezama, Jonathan Huang. Grant Schindler, Rachel Hornung, Vighnesh Birodkar, Jimmy Yan. Ming-Chaan -hang Chiu, et al. Videopoet: large language model for zero-shot video generation. arXiv preprint arXiv:231..44225 2023.
Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong. Xin Li, et al. Hunyuanvideo: A systematic framework for large video generative models, 2024. URL https: org/absg////ooo 2412.03603
Zhenglun Kong, Yize Li, Fanhu Zeng. et al. Token reduction should go beyond efficiency in generative models from vision, language multimodality. arXiv preprint arXiv:2505. /8227, 2025.
Chengxuan Li, Di Huang, Zeyu Lu, Yang Xiao, Qingqi Pei, and Lei Bai. A survey on long video generation: Challenges, methods, and prospects. arXiv preprint arXiv:2403. /6407, 2024a.
Yanyu Li, Pu Zhao, Geng Yuan, Xue Lin, Yanzhi Wang, and Xin Chen. Pruning-as-search: Efficient neural architecture search via channel pruning and structural reparameterization. arXiv preprint arXiv:2206.01/ /98, 2022.
Yanyu Li. Changdi Yang, Pu Zhao, et al. Towards real-time segmentation on the edge. 2023a. ISBN 978-1-57735-888 doi: 10. 00.1609/aaaivv3722.55252 URL httppp:///o..oggopoepottt
MAAT23/AAA
1609 aaai 23312.25233
Yize Li, Pu Zhao, Xue Lin, Bhavya Kailkhura, and Ryan Goldhahn. Less is more: Data pruning for faster adversarial training. arXiv preprint A.Kv:22302./3366 2023b. 2023b..
Yize Li, Pu Zhao, Ruyi Ding, Tong Zhou, al. Neural architecture search for adversarial robustness via learnable pruning. Frontiers in High Performance Computing, 2024b.
Yize Li, Lei Lu, Zhenglun Kong, al. Diff-stygs: 3d gaussian splatting stylization via tuning vulti-view sparse diffusion. OpenReview, 2025a.
Yize Li, Yihua Zhang, Sijia Liu, and Xue Lin. Pruning then reweighting: Towards data-efficient training of diffusion models. ICASSP. 2025b.
Yuhong Li, Yingbing Huang, Bowen Yang. Bharat Venkitesh, Acyr Locatelli, Hanchen Ye, Tianle Cai, Patrick Lewis, and Deming Chen. SnapKV: LLM knows what you are looking for before generation. In The Thirty-eighth Anmual Conference on Neural Information Processing Systems, 2024c. peattps://oppnreviieee http://www.e.etooooedddddeddeee.
net/
Yingyu Liang, Jiangxuan Long, Zhenmei Shi, Zhao Song, and Yufa Zhou. Beyond linear ap- proximations: A novel pruning approach for attention matrix. In The Thirteenth International Conference on Learning Representations, 2025. URL
net/ forum?
id= 8Pxwie
Bin Lin, Yunyang Ge, Xinhua Cheng, Zongjian Li, Bin Zhu, Shaodong Wang. Xianyi He, Yang Ye, Shenghai Yuan, Liuhan Chen, et al. Open-sora sora-sorappe plan: Open-source large video generation model. arXiv preprini 13X0v22412.00131, 2024a.
Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. Awq: Activation-aware weight quantization for Ilm compression and acceleration. In MLSys, 2024b.
Akide Liu, Jing Liu, Zizheng Pan, Yefei He, Gholamreza Haffari, and Bohan Zhuang. Minicache: Ky cache compression in depth dimension for large language models, arXiv preprint arXiv:2405.1433 2024.
Jun Liu, Shulin Zeng, Li Ding, Widyadewi Soedarmadji, Hao Zhou, Zehao Wang, Jinhao Li, Jintao Li. Yadong Dai, Kairui Wen, Shan He, Yaqi Sun, Yu Wang. and Guohao Dai. Flightygm: Efficient video generation model inference with online sparsification and hybrid precision fpgas. In Proceedings of the 2025 ACM/SIGDA International Symposium on Field Programmable Gate Arrays. FPGA 255 pp. 2-13, New York, NY, USA. 2025. Association for Computing Machinery. ISBN 979840071 3965. doi: 10.1145/3706628.. 05015777706628.878844444 URL TRRLhtpa https: or...o./g/00.......1.11.. 7706628.37082
11

---

Published as a conference paper at ICLR 2026
Lei Lu, Yize Li, Yanzhi Wang, Wei Wang, and Wei Jiang. Hdcompression: Hybrid-diffusion image compression for ultra-low bitrates. PRICAI, 2025.
Zhuoyan Luo, Fengyuan Shi, Yixiao Ge, Yuju Yang, Limin Wang, and Ying Shan. Open-magvit2: An open-source project toward democratizing auto-regressive visual generation. arXiv preprint arXiv:2409.04410, 2024.
Xinyin Ma, Gongfan Fang, and Xinchao Wang. LIm-pruner: On the structural pruning of large language models. In Advances in Neural Information Processing Systems, 2023.
Andrew Melnik, Michal Ljubljanac, Cong Lu, Qi Yan, Weiming Ren, and Helge Ritter. Video diffusion models: A survey. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. Survey Certification.
net forum? URL httpthtsss//pppeeeeeeeeeeeeeeeee htw.pppnnrrereeeeeeeeoorrrraaaaa rdd
Xupeng Miao, Gabriele Oliaro. Zhihao Zhang, Xinhao Cheng, Zeyu Wang, Zhengxin Zhang, Rac Ying Yee Wong, Alan Zhu, Lijie Yang. Xiaoxiang Shi, al. Specinfer: Accelerating generative large language model serving with tree-based speculative inference and verification. arXiv preprint arXiv:2305.09781, 2023.
Xuefei Ning. Zinan Lin, Zixuan Zhou, Zifu Wang. Huazhong Yang, and Yu Wang. Skeleton- of- of-thouuht: Prompting LLMs for efficient parallel generation. In The Twelfth International Conference on Learning Representations, 2024.
net forum?
idd=mgVVBbNNCm))
William Peebles and Saining Xie. Scalable diffusion models with transformers. arXiv preprint
arXiv:2212..09488, 2022.
Xiangyu Peng, Zangwei Zheng, Chenhui Shen, al. Open-sora 2.0: Training a commercial-level video generation model in $200k. arXiv preprint arXiv:2503.06644 2025.
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. 2019.
Isaac Rehg. Kv-compress: Paged Paged comp--ee with variable compression rates per attention head, 2024. URL htttp:://arxi..o///ppp/// hwtp://gg/bbb//aa/a 200.0//041100110111111
Xuan Shen, Yaohua Wang, Ming Lin, Yilun Huang, Hao Tang, Xiuyu Sun, and Yanzhi Wang. Deepmad: Mathematical architecture design for deep convolutional neural network. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6163-6173, June 2023.
Xuan Shen, Peiyan Dong, Lei Lu, Zhenglun Kong, Zhengang Li, Ming Lin, Chao Wu, and Yanzhi Wang. Agile-quant: Activation-guides quantization for faster inference of 11ms on the edge. In AAAI, 2024a.
Xuan Shen, Zhaoyang Han, Lei Lu, Zhenglun Kong, Peiyan Dong, Zhengang Li, Yanyue Xie, Chao Wu, Miriam Leeser, Pu Zhao, Xue Lin, and Yanzhi Wang. Hotaq: Hardware oriented token adaptive quantization for large language models. IEEE Transactions Computer-Aided Design of Integrated Circuits and Systems, pp. 1-1, 2024b. doi: 10. 10.1109/TCAAD222444487777
Xuan Shen, Pu Zhao, Yifan Gong, Zhenglun Kong, Zheng Zhan, Yushu Wu, Ming Lin, Chao Wu, Xue Lin, and Yanzhi Wang. Search for efficient large language models. In NewrIPS, 2024c.
Xuan Shen, Chenxia Han, Yufa Zhou, Yanyue Xie, Yifan Gong, Quanyi Wang. Yiwei Wang, Yanzhi Wang, Pu Zhao, and Jiuxiang Gu. Draftattention: Fast video diffusion via low-resolution attention guidance. arXiv preprint arXiv:2505.1 14708, 2025a.
Xuan Shen, Zhao Song, Yufa Zhou, Bo Chen, Yanyu Li, Yifan Gong, Kai Zhang, Hao Tan, Jason Kuen, Henghui Ding, et al. Lazydit: Lazy learning for the acceleration of diffusion transformers. In AAAI, 2025b.
12

---

Published as a conference paper at ICLR 2026
Xuan Shen, Zhao Song, Yufa Zhou, Bo Chen, Jing Liu, Ruiyi Zhang, Ryan A. Rossi, Hao Tan, Tong Yu, Xiang Chen, Yufan Zhou, Tong Sun, Pu Zhao, Yanzhi Wang, and Jiuxiang Gu. Numerical pruning for efficient autoregressive models. Proceedings the AAAI Conference on Artificial Intelligence, 39(19):20118-20422 Apr. 2025c. doi: 10. 16.9/aaai w3t..9..9999222.. URL https //ojs. http:aaa....rr...neeee.paa/aa/aa
34249
Xuan Shen, Yizhou Wang, Xiangxi Shi, Yanzhi Wang. Pu Zhao, and Jiuxiang Gu. Efficient reasoning with hidden thinking. arXiv preprint arXiv:2501. /9201, 2025d.
Xuan Shen, Hangyu Zheng, Yifan Gong, et al. Sparse learning for state space models on mobile. In ICLR, 2025e. httL:htts:///ot
net/ffrum? d=t.mm FNwn
Benjamin Spector and Chris Re. Accelerating IIm inference with staged speculative decoding. arXiv preprint arXiv:2308. 04623, 2023.
Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan. Autoregressive model beats diffusion: Llama for scalable image generation. arXiv preprini arXiv:2406.06525, 2024.
Yao Teng. Han Shi, Xian Liu, Xuefei Ning, Guohao Dai, Yu Wang, Zhenguo Li, and Xihui Liu. Ac- celerating auto-regressive text-to-image generation with training- speculative jacobi decoding. arXiv preprint arXiv:2410.01699 2024.
Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng, and Liwei Wang. Visual autoregressive modeling: Scalable image generation via next-scale prediction. 2024.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv: 13971, 2023.
Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang. Zhen Li, Qiying Yu, et al. Emu3: Next-token prediction is all you need. arXiv preprint arXiv:2409. 18869, 2024a.
Yuqing Wang, Shuhuai Ren, Zhijie Lin, Yujin Han, Haoyuan Guo, Zhenheng Yang. Difan Zou, Jiashi Feng, and Xihui Liu. Parallelized autoregressive visual generation. arXiv preprint arXiv:2412.15119 2024b.
Wenming Weng. Ruoyu Feng, Yanhui Wang, Qi Dai, Chunyu Wang, Dacheng Yin, Zhiyuan Zhao, Kai Qiu, Jianmin Bao, Yuhui Yuan, Chong Luo, Yueyi Zhang. and Zhiwei Xiong. Artev: Auto- regressive text-to-video generation with diffusion models. arXiv preprint arXiv:2311. 188834, 2023.
Yecheng Wu, Zhuoyang Zhang, Junyu Chen, Haotian Tang, Dacheng Li, Yunhao Fang, Ligeng Zhu, Enze Xie, Hongxu Yin, Li Yi, et al. Vila-u: unified foundation model integrating visual understanding and generation. arXiv preprint ar arXiv:2409. 240.0229999444,,, 2024.
Yushu Wu, Yifan Gong, Pu Zhao, et al. Compiler-aware neural architecture search for on-mobile real-time super-resolution. In ECCV, pp. 92-111. Springer, 2022.
Guangxuan Xiao. Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han. SmoothQuant: Accurate and efficient post-training quantization for large language models. Proceedings of the 40th International Conference on Machine Learning, 2023a.
Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. Efficient streaming language models with attention sinks. arXiv, 2023b.
Desai Xie, Zhan Xu, Yicong Hong, Hao Tan, Difan Liu, Feng Liu, Arie Kaufman, and Yang Zhou. Progressive autoregressive video diffusion models. arXiv preprint arXiv:2410.08/5/,, 2024.
Zhen Xing, Qijun Feng, Haoran Chen, Qi Dai, Han Hu, Hang Xu, Zuxuan Wu, and Yu-Gre Yu-Gang Jiang. A survey video diffusion models. ACM Computing Surveys, 57(2):144) 2024.
13

---

Published as a conference paper at ICLR 2026
Jing Xiong, Gongye Liu, Lun Huang, Chengyue Wu, Taiqiang Wu, Yao Mu, Yuan Yao, Hui Shen, Zhongwei Wan, Jinfa Huang, et al. Autoregressive models in vision: A survey. arXiv preprint arXiv:2411.05002, 2024.
Ruyi Xu, Guangxuan Xiao, Haofeng Huang, Junxian Guo, and Song Han. Xattention: Block sparse attention with antidiagonal scoring. arXiv preprini 1643,, 2025.
Changdi Yang, Pu Zhao, Yanyu Li, et al. Pruning parameterization with bi-level optimization for efficient semantic segmentation on the edge. In CVPR, 2023a.
Changdi Yang, Zheng Zhan, Ci Zhang, et al. Fairsmoe: mitigating multi-attribute fairness problem with sparse cexture-of-experre IJCAI, 2025.
Nan Yang, Tao Ge, Liang Wang, Binxing Jiao, Daxin Jiang, Linjun Yang, Rangan Majumder, and Furu Wei. Inference with reference: Lossless acceleration of large language models. arXiv preprint arXiv:2304. 04487, 2023b.
Zhuoyi Yang. Jiayan Teng, Wendi Zheng, Ming Ding, et al. Cogvideox: Text-to-vider diffusion models with an expert transformer. arXiv preprint arXiv:2408.06// 2024.
Zheng Zhan, Yifan Gong, Pu Zhao, Geng Yuan, al. Achieving on-mobile real-time super-resolution with neural architecture and pruning search. In /CCV, pp. 4821-4831, 2021. 2021.
Zheng Zhan, Zhenglun Kong, Yifan Gong, et al. Exploring token pruning in vision state space models. cdx
In NeurIPS, 2024a. URL httllttpps///eeeeeeeeeeeeeee///p
Zheng Zhan, Yushu Wu, Yifan Gong, et Fast and memory-efficient video diffusion using streamlined inference. In NeurIPS, 2024b. UNLhttps::/oppereevl.w htt. mettfoormmii
iNvXYOrkpi
Zheng Zhan, Yushu Wu, Zhenglun Kong, et al. Rethinking token reduction for state space models. In EMNLP, pp. 1686-1697, Miami, Florida, USA, 2024c. ACL. URL https
//aclantho wow.oolgy..ooo wttp://///2.2 emn mommlp-mmin.... 1mp.amn.1000
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as perceptual metric. In CVPR, 2018.
Yihua Zhang, Yuguang Yao, Parikshit Ram, al. Advancing model pruning via bi-level optimization. NeurlPS, 2022.
Pu Zhao, Xuan Shen, Zhenglun Kong, Yixin Shen, Sung- En Chang, Arash Akbari, Timothy Rup- precht, Lei Lu, Enfu Nan, Changdi Yang, et al. 7b fully open source moxin-llm/vlm from pretraining to grpo-based reinforcement learning enhancement. arXiv preprint 2rX1v:2412.06845, 2024a.
Pu Zhao, Fei Sun, Xuan Shen, et al. Pruning foundation models for high accuracy without retraining. In Findings of EMNLP 2024. ACL, 2024b. doi: 10.116533v1/2024 h/t.:///..//2.............. andings-emnlp.ee URL
a0dggg/o........ggoo.mmmm....m 566
Zangwei Zheng, Xiangyu Peng, Tianji Yang, Chenhui Shen, Shenggui Li, Hongxin Liu, Yukun Zhou, Tianyi Li, and Yang You. Open-sora: Democratizing efficient video production for all. arXiv 2024.
preprint ar arXiv:2412.
Junchen Zhu, Huan Yang, Wenjing Wang, Huiguo He, Zixi Tuo, Yongsheng Yu, Wen-Huang Cheng, Lianli Gao, Jingkuan Song. Jianlong Fu, et al. Mobilevidfactory: Automatic diffusion- social media video generation for mobile devices from text. In Proceedings of the 31 ACM International Conference on Multimedia, pp. 9371 1-13319 2023.
14

---

Published as a conference paper at ICLR 2026
APPENDIX
A ADDITIONAL RESULTS
A.I DETAILED RESULTS FOR VBENCII
We provide the detailed scores of VBench in Table 3 and Table 4 Our method better maintains the generation quality than SA methods. Specifically, on the VBench, when increasing the sparsity, unlike SA method with significant drop on most of subtasks, our method keeps high scores close to dense model on most subtasks under various sparsity.
Table 3: Detailed VBench scores.
Method
Replay Ratio
Local Size
Overall Consistency
Subject Consistency
Background Consisiency
Temporal Flickering
Motion Smoothness
Dynamic Degree
Aesthetic Quality
Imaging Quality
Dense
27.9%
87.8%
94.6%
95.8%
94.9%
59.4%
57.4%
58.8%
84.5%
100.0%
54.2%
Sparse Alln.
32
27, 9%
66.8%
86.2%
82.5%
82. 82.3%
100.0%
54.2%.
59.0%
27.7%
84.4%
128
27.5%
63.7%
85.1%
84.4%
84.3%
99.7%
52.9%.
57.2%
256
10%
27.7%
86.2%
94.0%%
95. 95.9%
94.6%
57.8%
57 57.0%
57.5%
27.9%
86.7%
93.7%
95.5%
57.5%
56.9%
57.4%
Ours
30%
27.9%
85.7%
93.4%
95.1%
94.4 94.4%
52.8%
57.0%
57.1% 1.6
40%
27.9%
93.7%
95.0%
94.9%
57.2%
57.5%
50%
27.9%
88.2%
94.1%
95.1%
95.0%
20.6%
57.7%
57.8%
27.9%
94.2%
94.9%
12.8%
57.9%
58
70%
29.1%%
89.4%
94. 14%
94.4%
95.0%
11.4%
58.2%
58.1%
28.2%
88.7%
93.8%
93.7%
94.4%
23.3%
58.2%
58.5%
Table 4: Detailed VBench scores.
Method
Replay Ratio
Local Size
Object Class
Multiple Objects
Human Action
Color
Spatial Relationshp
Scene
A Appearance Style
Temporal Style
Dense
/
76.7%
30.8%
74.8%
82.0%
37.6%
42.6%
24.7%
25.0%
16
67.8%
20.5%
81.8%
81.7%
41.2%
37.3%
24.6%
24.9%
Sparse Attn.
83.2%
37.2%
35.0%
24.5%
64
61.9%
17.2%
81.8%
77.6%
30.9%
33.7%
24.5%
25.2%
128
50.1%
78.8%
19.9%
32.6%
256
68.1%
26.6%
80.4%
76.7%
31.8%
38.9%
24.5%
25.3%
10%
74.9%
33.7%
76.8%
78.6%
42.2%
40.4%
24.7%
24.9%
20%
74.0%
34.0%
76.0%
78.9%
41.0%
40.6%
24.7%
25.1%
Ours
30%
74.4%
32.8%
75.8%
78.0%
39.8%
40.8%
24.8%
25.1%%
40%
76.5%
31.0%
73.8%
76. 19::
40.9%
40.6%
24.7%
25.2%
50%
77.1%
34.3%
74.0%
76.8%
42.7%
42.0%
24.8%
25.2%
60%
77.6%
36.0%
75.4%
70%
77. 97.9%
36.3%
79.0%
79.2%
43.8%
42.6%
24.8%
25.3%
A.2 DETAILED ABLATION RESULTS FOR THRESHOLD DISTRIBUTION
We provide full results for the ablation of threshold distribution in Table 5 We observe that consistent threshold achieves better performance with lower LPIPS and higher VBench score than inconsistent thresholds, which verifies the effectiveness of Remark 4.7 discussed in Section 4.4
Inconsistent Thresholds. The experimental results in Figure 4 include both consistent threshold and inconsistent threshold settings. However, we emphasize that the inconsistent threshold setting used in our experiment is not truly adaptive thresholding strategy. In our setup, the inconsistent thresholds were manually assigned across layers according to the replay ratio distributions as shown in Figure 5 This configuration mimics uneven thresholding scheme but designed to maintain the same overall replay ratio as the consistent threshold baseline, allowing for controlled comparison. Although they inconsistent, they still fixed, and not optimal adaptive.
genuinely adaptive thresholding mechanism, which dynamically adjusts thresholds based on token importance (e.g., derived from attention score distributions), may potentially lead to better
15

---

Published as a conference paper at ICLR 2026
performance than consistent threshold. Our work focus exploring the temporal redundancy in MLP modules using a plug-and-ple and-play cache replay strategy with consistent threshold simplifying deployments and achieving better performance. We leave the promising direction explore more- advanced aitention-wwar adaptive thresholds as our future work.
Performance of Inconsistent Thresholds. In (left, observe that according the LPIPS score for the inconsistent thresholds, the performance initially improves as the replay ratio increases, and eventually degrades very high replay levels. We highlight that since inconsistent thresholds not optimal or adaptive as discussed above, we still recommend to use consistent threshold with better performance. This trend of inconsistent thresholds can be explained by the fact that our proposed method helps reduce temporal drifting in the generated videos, as discussed in Section 6.4
More specifically, moderate replaying of MLP outputs introduces temporal consistency across frames, which helps suppress Trameto--frame inconsistencies and improves perceptual similarity. This effect visually evident in Figure [ where our method exhibits minimal drifting compared to sparse attention-based baselines, which suffer from severe drift artifacts. Furthermore, when our method is combined with sparse attention, it can help mitigate the drifting effects introduced by sparse attention, leading to improved overall visual coherence in the generated videos.
However, as the replay ratio becomes too large, over-replaying leads to excessive reuse of stale information, which degrades generation quality. Thus, there exists optimal replay ratio where replay enhances consistency without sacrificing content fidelity, explaining the observed trend in LPIPS.
A.3 DETAILED ABLATION RESULTS FOR THRESHOLD VALUES
We provide full results for the ablation of threshold values in Table 6 When S 2.5,, if we continue to decrease т, the generation quality does not further degrade while higher sparsity with faster inference is achieved, demonstrating the robustness of FastCar. Additionally, we observe that the AR video generation model achieves the highest replay ratio 87% when & -8, indicating that only 13% of the MLP modules are actually required during the generation process.
A.4 FULL RESULTS FOR ADDITIONAL ANALYSIS
We provide full results for the combination of the sparse attention method and our method in Table 7 The results show that method significantly boosts the performance of SA method through the straightforward combination. This validates the effectiveness of our method as complementary
enhancement existing SA approaches.
Table 5: Full results for the ablation of the threshold distribution.
Threshold Distribution
Replay Ratio
PSNR t
SSIM
LPIPS 4
VBench Score
Total
Quality
Semantic
Consistent
10%
18.57
53.32
27.31
73.4%
75.5%
65.2%
Inconsistent
10%
16.73
46.63
32.94
71.8%
73.7%
64.3%
Consistent
20%
17.94
51.01
27.57
73.2%
75.3%
65.1%
Inconsistent
20%
16.30
45.05
33.60
71.7%
73.3%
65.4%
Consistent
30%
17.87
50.29
28.02
72.4%
74.3%
64.7%
Inconsistent
30%
16.67
45.39
31.96
72.5%
74.0%
66.5%
Consistent 40%
17.68
50.14
28.15
71.8%
73.0%
67 27.2%
Inconsistent
40%
17.34
48.61
30.65
71.8%
73.6%
64.5%
Consistent
50%
17.85
50.11 11
28.08
71.5%
72.7%
66.6%
Inconsistent
50%
17.65
49.94
29.40
71.5%
73.0%
65.4%
Consistent 60%
17.85
50.55
28.72
71..%%
72.7%
66.2%
Inconsistent
60%
17.79
49.55
28.56
71.3%
72.5%
66.5%
Consistent 70%
17.86
50.18
28.79
71.2%
72.3%
66 66.9%
Inconsistent
70%
17.68
29.39
71.1%
72.4%
65.9%
Consistent
80%
17.71
49.0
29.50
71.5%
73.0%
65.66
Inconsistent
80%
17.59
30.01
71.3%
72.5%
66.3%
16

---

Published as a conference paper at ICLR 2026
Table 6: Full results for the ablation of the threshold values.
Threshold Values
Replay Ratio
PSNR
SSIM
LPIPS
VBench Score
Total
Quality
Semantic
0
3.96%
19.71
57.66
24.14
73.7%
75.7%
65.8%
-0.5
9.13%
18.61
53.49
27.27
73.6%
75.6%
65.5%
-1
17.32%
17.84
50.49
29.22
73.0%
75.1%
64.6%
-1.5
27.81%
17.31
48.39
30.85
72.6%
74.5%
64.7%
-2
40.92%
17.38
48.84
30.45
71.9%
73.6%
65.2%
-2.5
54.55%
17.76
50.30
29.05
71.4%
72.7%
65.8%
-3
66.78%
17.87
50.42
28.69
71.3%
72.5%
66.3%
3355
76.20%
49.42
71.3%
72.6%
66.2%
-4
82.41% %
17.65
48.51
29.83
71.4%
72.7%
66.2%
-8
87.49%
17.60
48.50
30.09
71.5%
72.9%
66.0%
-16
87.49%
17.60
48.04
30.09
71.5%
72.9%
66.0%
Table 7: Full results for the combination of the sparse attention method and our method.
Method
Threshold Value
Replay Ratio
Local Size
PSNR
SSIM
LPIPS
VBench Score Quality
TFLOPs
Latency (6)
Power
Total
Semantic
Etü.
Dense
74.1%
76.4%
65.0%
31.79
689.71
Ours
17.72%
16
12.96
29.57
55.13
60.8
61.6%
$7.8%
28.05
497.35
2.03
-2
46.11%
16
14.38
36.14
47.50
64.7%
66.5%
57.5%
23.69
427.75
2.36
Sparse
70.23%
34.25
70.5%
64.9%
356.35
Attn.
83.855 4
16
17.27
46.49
32.37
71.6%
73.1%
65.7%
17.87
331.29
3.05
Ours
-1
17.72%
32
13.25
31.58
53.77
61 22%
61. 9%
58. 2%
28.07
499.79
2.02
46.11%
14.43
47.11
65.00 0%
59.0%
23.71
430.18
Sparse
3
70.23%
32
16.94
46.02
34.09
70.8%
72.3%
64.9%%
19.83
358.7%
2,82
Attn.
31.96
Ours
17.72%
64
13.25
31.90
54.32
60.0%
60.9%
56.5%
28.10
504.60
2,00
-2
11%
64
14.41
36.83
47.72
64 666
66.2%
58 4%
23.74
435 00
2.33
Sparse
70.23%
16.88
45.87
34.60
70.44%
71.8%
64.7%
363.60
Attn.
4
83.85%
64
17.27
46.70
32.09
71.6%
73.1%
63.5%
17.92
338.55
2.99
Ours
-1
17.72%
128
13.14
31.63
55.02
59.1% 1%
60.3%
53.9%
28.16
514.00
1.97
46.11%
128
14.44
37.20
47.80
64.2%
65.9%
57.6%%
23,80
444.40
2.28
Sparse
70.23%
128
16.89
46.02
34.77
70.0%
71.5%
63.9%
19.92
373.00
2,71
Atin.
83.85%
17.29
32.10
71.7%
65.7%
347.95
Ours
-1
17.72%
256
15.22
40.77
44.61
67.7%
70.0%
58.7%
28.28
$31.87
1.90
46.11%
256
15.78
43,05
40.25
9%
71.1%
60.11
23,92
462.27
2.19
Sparse
70.23%
256
17.21
47.63
32.94
70.7%
72.2%
64.8%
20.04
390.87
2.59
Atin.
83.85%
256
17.44
47.57
31.27
71.8%
73.3%
65.7%
18,10
365 82
2.76
17

---

Published as a conference paper at ICLR 2026
B ADDITIONAL VISUALIZATION
We visualize the results of our method under different replay ratios. Our method generates high quality videos.
Waves rolling on the sea.
Buselin
17% Replny
Raplay
-2
Replog
A cat wearing sunglasses sitting on chair.
Bascline
12% Replay
-2
40% Roplag
82% 82% Replny
dog wearing sunglasses the beach.
Baseline
17% Replay
40% Roplay
Replay
Figure 7: Additional visualization with threshold 7 -1, -1,-2, -4.
18

---

Published as a conference paper at ICLR 2026
C DETAILED PROOFS
C.I. PROOF OF THEOREM
Proof Theorem 4.4 Step 1 (Score exactly matches cosine similarity). By Definition 4.3 81 (q kk))k////// Vd, where qj 2j We and k-- - Wx. Under Assumption (3), 1g k so St.i (up to vd scaling) equals the cosine similarity:
cos (gg; k--))) = (gjz k-z)-
Thus, by the Law of Cosines for unit vectors,
|1g - kj- 112 = 2(1 - st,1).
Step 2 (Logit gap from query gap). The attention logits satisfy
{ = 9j K l = KJ
thus
|e, 4-1122 = |1(a; - 9j- ) K 1| < E|K|e 12 45 1|
where K = XWW is the stacked key matrix. Since K = XWWx, we have |||| S | S VMMA, where | 12 E VnM since each ||= I ...
Step 3 (Attention output is Lipschitz). Since softmax and value projection are Lipschitz continuous (see Shen et al. (2025b), there exists Taam > such that
||Attn(X)) )4:- Att -.ll2 S Lamulee - Lj 1l2 S Cilla, - q 1l2.
where C1 = Truin MAA.
Step 4 (Bounding query-key difference). Since
qj Ij Wo, k-= =I, WK.
it follows that
||:-- 9j- 2|2 = || (WK Wo)lla S ~11 ll2 S MM.
By triangle incquality.
|/u Go 1l2 S 11 - k5- ||kg- - 10- 112 S 20 81,1) + 7.
Step 5 (Final bound). Thus,
[Attn(X),, Attn(x))_1lla30) (V211 s1,s) + YM
SC((T- 81,5 + 7M)
after absorbing constants into C. > 0. This completes the proof.
C.2 PROOF OF THEOREM 4.5
Proof of Theorem 4.5
Define
Z, = Attn(X),, + Xy.:.
Zy- = Attn(X); Xj-
Then
Yj.- = MLP((2),
Yz= MLP(ZZ-).
By Lipschitz continuity of MLP.
Iv: Y S T.I 7 1|2.
Expanding Z, Z, and applying triangle inequality,
12, Z,- lle < ||ttn(X)), Attn(X)) + || X1-
The claim follows.
19

---

Published as a conference paper at ICLR 2026
C.3 PROOF OF THEOREM 4.6
Proof of Theorem 4.6
By Theorem 4.5
1 Y-,lll S I. (11X xz-,1l2 + |Attn(X),, Attn(X))-,111))
By Theorem 4.4
||ttt((X)s: - A1 C" (VI- 31,5 + 7.1)
Substituting gives
[} - Yg-..la S Xy- + v - 81, + +7M),
where C' L(1 + C*) absorbs constants.
20


Reviews:
[{"review_id": "PYExEZJcRi", "reviewer": "Reviewer_XRWX", "summary": "This paper aims to address the high computational cost of auto-regressive (AR) video generation, particularly for deployment on edge devices. The authors argue that in AR video generation, the MLP modules—not the attention modules—dominate the inference latency during the decoding phase. They also observe a high degree of temporal redundancy in the outputs of these MLP modules for adjacent frames.\n\nBased on these findings, they propose FastCar, a framework that accelerates decoding by conditionally reusing cached MLP outputs from the previous frame. The decision to reuse or recompute is governed by a novel metric called the Temporal Attention Score (TAS), which measures the similarity between a token's query and the key of its corresponding token in the prior frame. \n\nExperimental results on the VILA-U model show that FastCar significantly outperforms sparse attention methods, achieving over a 2.1x speedup when combined with sparse attention, while better preserving generation quality and improving energy efficiency.", "questions": "1. The Temporal Attention Score (TAS) is calculated by averaging scores across all attention heads. Did you investigate the behavior of individual heads? Is it possible that some heads are more specialized in tracking temporal correspondence and could serve as better indicators for the replay decision than a simple average?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. The paper is well-written. The proposed method is well-illustrated and easy to follow.\n\n2. **The proposed method is simple and efficient:** Using the Temporal Attention Score (TAS)—a metric that is effectively \"free\" as it is derived from pre-existing attention calculations—to guide the caching strategy is a very clever design choice. This avoids the overhead that often plagues other dynamic execution methods.\n\n3. **Comprehensive empirical evaluation:** The paper presents a robust set of experiments with convincing results. The method demonstrates significant improvements in latency, throughput, and power efficiency. The comparison against sparse attention (StreamingLLM) effectively highlights the superiority of the proposed approach for this problem domain. Furthermore, showing that FastCar can be combined with sparse attention to mitigate its weaknesses (like \"drifting\") and achieve even greater speedups is a powerful demonstration of its complementary nature.\n\n4. **Practical hardware co-design and validation:** This work goes beyond a purely algorithmic proposal by designing, implementing, and evaluating a hardware accelerator on an FPGA. The development of Dynamic Resource Scheduling (DRS) to handle the dynamic workloads shows a deep understanding of the practical challenges involved in deploying such models. This end-to-end, software-hardware co-design approach greatly increases the credibility and practical value of the research.", "weaknesses": "1. **Limited architectural generalization:** The experiments and analysis are conducted exclusively on the VILA-U model. While the authors correctly note the lack of other open-source AR video models, this raises a question about the generality of the core insight. The MLP-bottleneck observation may be specific to this particular architecture's configuration (e.g., hidden size, MLP expansion factor). The paper would be stronger if it included analysis on other AR models (even from image or language domains) to hypothesize about the broader applicability of the findings.\n\n2. **Static thresholding mechanism:** The replay decision relies on a single, manually-tuned, and globally consistent threshold (`τ`). While effective, this is a relatively simple criterion. A more sophisticated, adaptive thresholding mechanism could potentially unlock a better trade-off between performance and quality. The ablation on \"inconsistent thresholds\" is a good first step, but since the thresholds were manually set, it doesn't fully explore the potential of adaptive strategies.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "omqcB8biK4", "reviewer": "Reviewer_RrDJ", "summary": "This paper proposes FastCar, a system-level framework to accelerate auto-regressive (AR) video generation on edge devices by exploiting temporal redundancy in MLP outputs across adjacent video frames. The core insight is that MLP modules, not attention, dominate decoding latency in AR video models like VILA-U, and their outputs exhibit high similarity between consecutive frames. To leverage this, the authors introduce the Temporal Attention Score (TAS), a lightweight metric derived from attention logits, to decide when to replay (i.e., reuse) cached MLP outputs from the previous frame instead of recomputing them. They further design a custom FPGA-based hardware accelerator with Dynamic Resource Scheduling (DRS) that adapts computation allocation in real time based on TAS-driven replay decisions. Experiments show that FastCar achieves >2.1x decoding speedup, ~45% FLOPs reduction, and higher energy efficiency than sparse attention baselines, while preserving video quality and even mitigating temporal drifting when combined with sparse attention.", "questions": "Generality beyond VILA-U: Is FastCar applicable to other AR video models that may have different MLP/attention ratios or tokenization schemes? What architectural properties are required for TAS to be predictive?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Strong empirical motivation: The paper provides compelling profiling evidence that MLPs, not attention, are the bottleneck in AR video decoding, which justifies shifting optimization focus away from KV-caching or sparse attention (common in LLMs) toward MLP replay.\nNovel and well-motivated algorithmic component: The Temporal Attention Score (TAS) is a simple yet effective proxy for temporal similarity that incurs zero extra compute (as it reuses existing attention logits). The theoretical analysis (Theorems 4.4–4.6) formally links TAS to MLP output similarity, lending credibility to the replay strategy.\n2. Hardware-software co-design: The integration of TAS with a custom FPGA accelerator featuring DRS is a notable strength. The DRS mechanism intelligently balances workload across cores in response to dynamic replay patterns, which is essential for real-world deployment.\n3. Comprehensive and reproducible evaluation: The authors evaluate across multiple replay ratios, report detailed metrics (PSNR, SSIM, LPIPS, VBench, latency, power, FLOPs), compare against StreamingLLM (a relevant sparse attention baseline), and demonstrate complementarity with sparse attention.\n4. Practical relevance: The work directly addresses a critical bottleneck in deploying AR video models on edge devices, making it highly relevant to both systems and generative AI communities.", "weaknesses": "1. Threshold selection is manual: The replay threshold is tuned empirically. The paper shows robustness across thresholds (Fig. 4), but does not propose an adaptive or learned thresholding strategy, which could improve usability in dynamic real-world scenarios.\n2. Limited evaluation and comparison: Evaluation limited to VILA-U, comparison limited to StreamingLLM. Yes, VILA-U is the only open-source AR video generation without diffusion model, but it may indicate that the community and business are not interested in such types of models.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vg5l5Jj7Q9", "reviewer": "Reviewer_wS7K", "summary": "This paper proposes FastCar, a method to speed up autoregressive video generation by avoiding redundant MLP computation across adjacent frames. The authors profile decoding latency and argue that the MLP/FFN block, not attention, is the main bottleneck. They observe that many tokens barely change between consecutive frames, so instead of recomputing the MLP output every time, FastCar “replays” cached MLP outputs from the previous frame for stable tokens. A Temporal Attention Score, derived from cross-frame attention, is used to decide which tokens can be safely replayed, and the paper provides a theoretical argument that high TAS implies similar MLP outputs.\n\nThe paper further presents an FPGA-oriented runtime scheduler (DRS) that dynamically assigns only the “needs recompute” tokens to compute cores to keep utilization high. Experiments claim >2× speedup over dense decoding and better quality / less drift than sparse-attention baselines. The high-level idea of reusing features for unchanged regions is conceptually simple and related to prior token-reuse / feature-caching work, but the authors position their novelty in applying it to autoregressive video decoding, tying the gating decision to TAS, and demonstrating a hardware-aware implementation.", "questions": "See Weaknesses", "limitations": "", "rating": 4, "confidence": 2, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The paper performs actual latency profiling and shows that, in autoregressive video decoding, the main bottleneck is the MLP/FFN block rather than attention. This is valuable because most prior acceleration work focuses on attention; here the motivation is concrete and data-driven.\n\nThe core mechanism of FastCar — reusing the previous frame’s MLP outputs for tokens that barely change, instead of recomputing them every step — is straightforward and can be applied at inference time without retraining. The per-token, per-layer replay design makes it easy to imagine plugging this into existing AR video generators as a decoding-time optimization.\n\nThe method uses a Temporal Attention Score (TAS), derived from cross-frame attention, to decide which tokens can safely skip recomputation. The paper also provides a theoretical argument that high TAS implies similar MLP outputs. This lifts the approach above a pure heuristic and makes the selective replay decision more interpretable.\n\nBeyond the algorithmic idea, the paper presents a hardware-oriented runtime scheduler (DRS) on FPGA that dynamically dispatches only the “needs recompute” tokens to available compute cores, improving utilization and energy efficiency. This gives the work a full-stack feel rather than just a conceptual trick.\n\nThe experiments report over 2× decoding speedup while maintaining generation quality and reducing long-horizon drift compared to sparse-attention baselines. This suggests the approach could be especially useful for long-duration or higher-resolution AR video generation.", "weaknesses": "1. The core idea — detect tokens/regions that barely change over time and skip recomputing them by reusing cached features — is not fundamentally new. Similar per-token reuse / feature caching / dynamic skipping ideas have already appeared in video and diffusion generation acceleration. The paper mainly adapts this intuition to autoregressive video decoding, adds a specific gating signal (TAS), and wraps it with an FPGA story. This feels more like an incremental systemization than a genuinely new algorithmic principle.\n\n2. And as written, all results (latency profiling, TAS analysis, replay ablations, quality vs. drift comparisons, and the FPGA/DRS speedup and energy numbers) appear to be reported on a single autoregressive video generation model (vila-u-7b-256)? Could you explain why the evaluation is limited to this one model? In particular: (1) is FastCar tied to architectural details of that specific model (e.g., tokenizer, decoder layout, MLP shape), or should it in principle apply to other AR video generators? (2) did you attempt to run the method on any other AR video backbones or model sizes, and if so, what prevented you from reporting those results? Right now it is difficult to judge how general the proposed approach is versus a case study on one model. \n\nIf I missed any details, please let me know during the rebuttal period.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "NXc29IWtSR", "reviewer": "Reviewer_jt8U", "summary": "This paper proposes FASTCAR, an efficient framework for auto-regressive (AR) video generation, based on the insight that MLP modules are the main latency bottleneck and exhibit high temporal redundancy. FASTCAR implements a \"cache attentive replay\" strategy, using a Temporal Attention Score (TAS) derived from the preceding attention layer at no extra computational cost to determine whether to skip an MLP computation and reuse the cached output from the previous frame. Experimental results demonstrate that this method outperforms sparse attention approaches on VBench, achieving speedup with better generation quality and power efficiency on edge hardware.", "questions": "Though QK^T is computed in the attention layer, the result is temporary and on-chip in Flash Attention implementations. How do you calculate TAS efficiently when Flash Attention fusion is used? Does this require kernel modifications and what is the overhead?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. This paper is well-written and easy to read.\n2. The proposed method is simple, efficient and effective.\n3. When combined with sliding window attention, FastCAR show the possibility to alleviate drifting caused by global information dropping.", "weaknesses": "1. The hardware design section is not connected to other parts and no experiment results prove its efficiency.\n2. The claim that MLPs dominate latency is based on 8 frames short sequences which is relatively short to real-world using cases.\n3. Lacks discussion of similar caching approaches, such as DeepCache in diffusion generation.\n4. typo: \"third\" in line 459. \n\n[1] [CVPR2024] DeepCache: Accelerating Diffusion Models for Free", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)