Title: LeMiCa: Lexicographic Minimax Path Caching for Efficient Diffusion-Based Video Generation
Venue: NeurIPS 2025
Accepted: yes
Keywords: Video Generation

Model Acceleration

Caching Mechanism

Training-Free
Abstract: We present LeMiCa, a training-free and efficient acceleration framework for diffusion-based video generation. While existing caching strategies primarily focus on reducing local heuristic errors, they often overlook the accumulation of global errors, leading to noticeable content degradation between accelerated and original videos. To address this issue, we formulate cache scheduling as a directed graph with error-weighted edges and introduce a Lexicographic Minimax Path Optimization strategy that explicitly bounds the worst-case path error. This approach substantially improves the consistency of global content and style across generated frames. Extensive experiments on multiple text-to-video benchmarks demonstrate that LeMiCa delivers dual improvements in both inference speed and generation quality. Notably, our method achieves a 2.9× speedup on the Latte model and reaches an LPIPS score of 0.05 on Open-Sora, outperforming prior caching techniques. Importantly, these gains come with minimal perceptual quality degradation, making LeMiCa a robust and generalizable paradigm for accelerating diffusion-based video generation. We believe this approach can serve as a strong foundation for future research on efficient and reliable video synthesis.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces LeMiCa, a novel, training-free, and model-agnostic framework for accelerating diffusion-based video generation. The key idea is to improve caching decisions by framing them as a global path planning problem using a Lexicographic Minimax Path Optimization approach. Rather than relying on local heuristics for cache selection, LeMiCa constructs a directed acyclic graph (DAG) where each edge represents a potential cache segment, weighted by its global error impact. This enables selection of caching paths that bound the worst-case degradation.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. Clear conceptual novelty: The formulation of caching as a global lexicographic minimax optimization problem is a well-justified and innovative leap from local greedy heuristics.\n2. Strong empirical results: Extensive experiments across three T2V models consistently show LeMiCa's advantages in LPIPS, SSIM, PSNR, and VBench metrics.\n3. Training-free and model-agnostic: The framework does not require architectural changes or fine-tuning, broadening its applicability.\n4. Scalability and robustness: Effective across varying resolutions, frame counts, and denoising trajectories.\n5. Well-organized experimental section, including ablations and robustness checks.\n\nWeaknesses:\n1. Error estimation remains heuristic: While global in scope, the edge weights used in the DAG are derived from averaging over prompts and may miss task-specific subtleties.\n2. No runtime overhead claim may underplay preprocessing: The offline DAG construction phase is significant, especially for real-time applications.\n3. Limited theoretical insight into optimization guarantees: The use of lexicographic minimax is intuitive, but theoretical convergence or optimality bounds are not discussed.\n4. No uncertainty quantification: Error bars/statistical significance are missing from results, which limits rigor.\n\nQUESTIONS:\n1. How sensitive is LeMiCa to the choice of prompts used for DAG construction?\nAlthough ablations are provided (e.g., 1–350 samples), can performance degrade under domain shifts, such as out-of-distribution prompts or styles?\n\n2. Why choose lexicographic minimax over simpler bounded-error strategies?\nCould a hybrid strategy (e.g., combining minimax with average-case bounding) perform better in certain quality-speed regimes?\n\n3. What is the overhead of the offline DAG construction in practice?\nPlease provide empirical numbers (e.g., time, memory) for building the cache DAG on large models like CogVideoX.\n\n4. Could LeMiCa generalize to adaptive or online caching?\nThe current setup is entirely static. In latency-constrained deployments (e.g., real-time rendering), could partial DAG updates or dynamic adjustments help?\n\n5. Missing error bars/statistical significance.\nGiven the strong claims of performance gains, please provide error bars or confidence intervals over multiple seeds or trials. This would improve confidence in reported improvements and potentially increase the final score.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes LeMiCa, a training-free acceleration framework for diffusion-based video generation that improves upon existing caching methods. The authors identify that \"local-greedy\" caching strategies, which make decisions based on adjacent timesteps, fail to control the global accumulation of errors. To solve this, LeMiCa formulates the cache scheduling process as a path optimization problem on a Directed Acyclic Graph (DAG). Each edge in the DAG represents a potential cache segment, weighted by its global impact on the final generated video output quality. The framework then uses a Lexicographic Minimax optimization strategy to find the optimal path that minimizes the worst-case error under a fixed inference budget. Experiments on state-of-the-art models like Open-Sora and Latte show that LeMiCa achieves significant speedups of up to 2.9x while better preserving video quality better compared to prior methods.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n-\tThe paper proposes a novel approach to the caching problem. Shifting from a local decision process to a global path optimization problem is a logical and effective idea. The formulation of cache scheduling as a lexicographic minimax problem on a DAG is also novel/interesting and addresses the issue of compounding errors.\n\n-\tThe experimental results are thorough and convincing. LeMiCa consistently outperforms SOTA caching methods like TeaCache across multiple models (Open-Sora, Latte, CogVideoX) and metrics. Table 1 shows LeMiCa achieving a better trade-off between speed and quality, and the qualitative results in Figures 3 and 4 visually confirm its superior editing quality as do the examples in the supplementary materials.\n\n-\tThe ablation studies are thorough. \n\nWeaknesses:\n\n-\tDoes the computational complexity of graph construction lead to any slowdown? Can a performance analysis over the cost of the graph construction also be done similar to the ablations provided on performance at different resolutions and lengths.\n\n-\tThe number of video examples provided in the supplementary are very limited (only 3), which leads to the possibility of the framework not being effective across a broad set of videos and only effective at maintaining quality on a small subset or cherry-picked examples.\n\nQUESTIONS:\nMy questions are listed in the weaknesses section.\n\nLIMITATIONS:\nyes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a training‑free framework for accelerating diffusion‑based video generators. Instead of re‑using UNet/DiT activations with simple “local‐greedy” rules (e.g., TeaCache, PAB), LeMiCa does the following: 1. Defines a “global outcome‑aware” error: how much a whole segment of cached steps changes the final decoded video, averaged over several prompts/seeds; 2. Builds an offline Directed Acyclic Graph (DAG) whose edges are candidate cache segments weighted by that global error; 3. Searches under a fixed inference‑budget B the path that lexicographically minimizes the worst‑case edge weight (the minimax path). At run‑time only those B timesteps are fully computed; all others reuse cached features, incurring virtually no online overhead.\n\nSTRENGTHS AND WEAKNESSES:\nStrength:\n\n-- The paper articulates a genuine failure mode visible in prior work, providing a clear diagnosis of “local‑greedy” caching weaknesses (temporal heterogeneity & error accumulation).\n\n-- The formulation is novel: casting cache selection as a global path search with a lexicographic minimax objective.\n\n-- The paper provides thorough ablations: effect of sample count for graph construction, minimax vs shortest path, trajectory scale, resolution & length scaling.\n\n\nWeakness:\n\n-- The biggest problem of this work is the lack of convincing qualitative results. Firstly, I can only play the open-sora video in the supplementary material, not the other two; secondly, only three examples are shown, where none of them has large, rigid motions, but rather static scene with small motions; thirdly, for the three simple examples, I still observe significant artifacts: several parts are blurred out, and there is significant motion artifacts, e.g. where a snowflake blends with the background. These examples are not convincing for me that the approach does not suffer from much quality degradation, especially when comparing with TeaCache.\n\nQUESTIONS:\nN/A\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nLeMiCa (Lexicographic Minimax Path Caching) is a new training-free acceleration method for text-to-video diffusion models. The core idea is to reuse intermediate model outputs (caching) while carefully controlling errors on a global scale. Unlike prior “local” caching heuristics that decide frame by frame, LeMiCa treats the whole generation process as a path through a directed acyclic graph (DAG) of diffusion timesteps. Each possible skip (cache segment) is an edge weighted by its impact on final output quality. The authors introduce a lexicographic minimax optimization to choose a caching path that minimizes the worst-case generation error under a fixed computation budget. This strategy explicitly bounds the largest error between the accelerated video and a full-quality video, preventing cumulative drift. In experiments on multiple video generation benchmarks, LeMiCa achieves significant speed-ups (e.g. ~2.9× faster on the Latte model) while maintaining high fidelity (e.g. LPIPS ≈0.05 on Open-Sora), outperforming previous caching techniques in both efficiency and visual consistency. The method requires no model retraining or architectural changes, highlighting its practical utility for faster diffusion-based video synthesis.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n\n- **Practical Impact:** LeMiCa offers a *model-agnostic, training-free* acceleration framework that can be applied to existing diffusion video generators without retraining. This makes it highly practical for real-world use, enabling much faster inference (2–3× speed-ups in tested cases) while introducing only minimal perceptual degradation. Such immediate efficiency gains will be valuable for deploying diffusion models in interactive or production settings.\n- **Robust Generation Quality:** The proposed lexicographic minimax strategy effectively controls error accumulation, leading to improved global consistency across frames. The results show that accelerated videos remain very close to originals in structure and detail (e.g., extremely low LPIPS error of 0.05), addressing a key weakness of prior caching methods that often caused content drift or loss of fine details. Notably, the paper demonstrates better structural alignment and style consistency than the state-of-the-art TeaCache baseline, indicating strong preservation of video quality even at high acceleration.\n- **Thorough Evaluation:** The submission includes extensive experiments on multiple text-to-video benchmarks and base models (such as Latte and Open-Sora). It consistently shows **dual improvements** – faster inference and higher generation fidelity – across the board. Ablation studies and analyses of error metrics are provided, giving credibility to the approach’s advantages. The empirical results are convincing and reinforce the significance of the proposed method.\n\n**Weaknesses:**\n\n- **Complexity and Generalization:** LeMiCa’s global DAG construction involves evaluating error impacts with multiple prompts and selecting an optimal path. This added complexity means implementing the method requires extra analysis offline. The authors should clarify how general the precomputed graph is, for instance, does it reliably cover arbitrary prompts and scenes, or could certain out-of-distribution videos see different error dynamics? This is a minor issue, but discussing the method’s generalization beyond the tested data would strengthen the work.\n\nQUESTIONS:\n- **Generality to New Content:** The approach builds a static error-weighted DAG using multiple prompts to estimate segment errors. How sensitive is this graph to the choice of calibration prompts or data domain? In other words, if one were to apply LeMiCa to very different videos or prompts not seen during graph construction, would the caching decisions remain effective? It would be helpful to discuss the method’s robustness to domain shifts or how one might update the cache strategy for new types of content.\n- **Combination with Other Accelerations:** Since LeMiCa is *training-free* and model-agnostic, have the authors considered combining it with other inference acceleration techniques (e.g., model distillation or quantization)? For instance, could one first apply LeMiCa to reduce steps and also use a distilled model for larger jumps, or would that risk compounding errors? A brief discussion on how this caching strategy might complement (or differ from) training-based accelerations could clarify its place in the broader landscape of efficient diffusion methods.\n\nLIMITATIONS:\nNo. The submission does not explicitly include a dedicated *limitations* section or discuss potential **societal impacts** of the work.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-11"}