Title: Plug-and-Play Context Feature Reuse for Efficient Masked Generation
Venue: NeurIPS 2025
Accepted: yes
Keywords: Non-autoregressive Image Generation; Masked Generative Modeling; Inference Acceleration;
Abstract: Masked generative models (MGMs) have emerged as a powerful framework for image synthesis, combining parallel decoding with strong bidirectional context modeling. However, generating high-quality samples typically requires many iterative decoding steps, resulting in high inference costs. A straightforward way to speed up generation is by decoding more tokens in each step, thereby reducing the total number of steps. However, when many tokens are decoded simultaneously, the model can only estimate the univariate marginal distributions independently, failing to capture the dependency among them. As a result, reducing the number of steps significantly compromises generation fidelity. In this work, we introduce ReCAP (Reused Context-Aware Prediction), a plug-and-play module that accelerates inference in MGMs by constructing low-cost steps via reusing feature embeddings from previously decoded context tokens. ReCAP interleaves standard full evaluations with lightweight steps that cache and reuse context features, substantially reducing computation while preserving the benefits of fine-grained, iterative generation. We demonstrate its effectiveness on top of three representative MGMs (MaskGIT, MAGE, and MAR), including both discrete and continuous token spaces and covering diverse architectural designs. In particular, on ImageNet256 class-conditional generation, ReCAP achieves up to 2.4$\times$ faster inference than the base model with minimal performance drop, and consistently delivers better efficiency–fidelity trade-offs under various generation settings.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nMasked Generative Models (MGMs) have shown impressive performance for image synthesis. This paper tackles a redundant operation in decoding steps and proposes a ReCAP, similar to the KV cache in LLMs. Grouped decoding pipeline offers a flexible trade-off between efficiency and quality. The method is compared on the ImageNet benchmark on common metrics across various MGMs.\n\nSTRENGTHS AND WEAKNESSES:\nStrenghts\n- The motivation and its analysis are clear.\n- The proposed method that extends KV cache to MGMs with a grouped decoding framework is reasonable.\n- The paper is well-written, and the visualization of motivation and method is well-made.\n\nWeaknesses\n- **Comparison with Recent Work.** Although the authors compare MaskGIT-r+ReCAP with various MaskGIT variants, recent work that utilizes learnable guidance [1] demonstrates superior FID-efficiency trade-offs, achieving an FID of 3.35 with 24 NFEs using MaskGIT, while MaskGIT-r+ReCAP achieves an FID of 3.98 with 22+10 NFEs. I wonder if ReCAP can be integrated with the approach in [1], or if not, the authors should provide a comparison between these methods.\n\n- **Ablation on $l_{u+1:u+T'}$**. While Fig.3 illustrates that $l_{u+1:u+T'}$ could be set to 2 or higher, the actual implementation uses a value of 1, in all experiments, including confidence-based sampling. \nGiven this, it would be valuable to analyze whether reducing the total sampling timestep is more beneficial than setting $l_{u+1:u+T'}$ to a value greater than 1. The paper lacks analysis on this aspect. For instance, which config shows better quality-efficiency using $(u,T,T')=(8,12,4)$ or $(0,8,16)$?\n\n[1] Hur, J., Lee, D., Han, G., Choi, J., Jeon, Y., & Kim, J. (2024). Unlocking the Capabilities of Masked Generative Models for Image Synthesis via Self-Guidance. Advances in Neural Information Processing Systems, 37, 130977-130999.\n\nQUESTIONS:\nThe parameters $u$, $T$, and $T'$ play a crucial role in managing the trade-off between quality and efficiency. However, in all experiments, these values are selected empirically without adequate justification. How are these parameters chosen, or is there a principled approach for selecting them? Reporting performance variations based on different hyperparameter settings would enhance the paper and provide a more comprehensive understanding of their impact.\n\nLIMITATIONS:\n- While Full and Local FE provide flexible trade-offs, the Local-FE steps ($l_{u+1:u+T'}$) are constrained to 1, in addition to the parameter $u$, which further limits the upper bound of efficiency improvements. Thus, the efficiency gain could be marginal, as it requires Full-FE for more than half of the total steps.\n- All experiments are conducted on 256 resolutions with class conditional generation. Although ImageNet 256 is a comprehensive dataset, conducting experiments on a larger resolution (512) or Text-to-Image (T2I) model, such as [2] will make the paper more complete.\n\n[2] Wu, S., Zhang, W., Xu, L., Jin, S., Wu, Z., Tao, Q., ... & Loy, C. C. (2025). Harmonizing visual representations for unified multimodal understanding and generation. arXiv preprint arXiv:2503.21979.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-10"}

comments: SUMMARY:
This paper introduces Reused Context-Aware Prediction (ReCAP), a plug-and-play module designed to accelerate inference in masked generative modeling (MGM). By reusing cached key-value (KV) pairs, ReCAP replaces costly full evaluation steps with computationally efficient ones. Experiments demonstrate its effectiveness across multiple representative MGM works.

STRENGTHS AND WEAKNESSES:
Strengths:
1. The paper is well written and easy to follow.
2. The idea of re-using KV cache is simple and effective.
3. Comprehensive experiments validate ReCAP’s improvements across three MGM methods.

Weaknesses:
I do not see much apparent weakness in this paper. However, the current design depends on manual heuristics to determine where ReCAP steps replace full FE steps. While effective, this limits generality.

QUESTIONS:
none.

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-06-27

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nReCAP is a method designed to accelerate masked generative models by reusing context features, achieving up to 2.4× faster inference with minimal quality degradation across different architectures and tasks. The authors present experimental results to support their claims, demonstrating improved speed with comparable generation quality. However, I have several concerns about the paper. First, the reported 2.4× speedup is achieved under optimal conditions, while the generation quality drops significantly in certain cases, which is unacceptable for practical use. Second, the effectiveness of ReCAP on masked text-to-image (T2I) models remains unexplored, leaving a critical gap in the evaluation. Overall, I find the contribution of the paper to be limited. The current method does not offer a substantial improvement in speed or quality, and additional revisions are needed. Enhancing the approach to deliver more consistent and competitive acceleration would be necessary for this work to be considered a significant contribution.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n\n1. The paper presents strong empirical evidence, such as feature similarity analysis, to support its key assumption regarding the stability of context features during decoding.\n\n2. The writing is clear and well-organized, complemented by intuitive figures and detailed implementation descriptions that enhance both understanding and reproducibility.\n\n3. Additional experiments provided in the appendix add depth and comprehensiveness to the paper.\n\n**Weaknesses:**\n\n1. The core idea bears similarities to TeaCache, but the paper neither cites nor discusses this related work, which could be seen as a significant oversight.\n\n2. Although ReCAP achieves up to a 2.4× speedup, this is not very competitive compared to one-step generators like Flux, which can generate images in a single step without the need for classifier-free guidance (CFG). Furthermore, masked generation models currently lag behind diffusion models in terms of generation quality, which limits the practical applicability of this approach.\n\n3. The paper lacks text-to-image (T2I) experiments. While I understand the method is designed to be lightweight and plug-and-play, it would significantly strengthen the work to demonstrate its effectiveness in the T2I setting. For example, applying it to a model like Meissonic.\n\n4. The results in Table 1 show a notable performance drop for MaskGIT when reducing the number of steps to 16, with the FID score worsening from 4.46 to 5.02. Additionally, the acceleration on MAR-H appears insufficient and may not justify the trade-off in performance.\n\nQUESTIONS:\nSee weaknesses.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-27"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis submission proposes an optimization for inference with Masked Generative Modelling inspired by KV-caching techniques used in Autoregressive models.\n\nThe key observation is that, in MGMs, the attention keys and values (KVs) of previously decoded (context) tokens evolve slowly across decoding steps, and thus could be approximated via caching.\nThe authors thus present ReCAP, an inference algorithm for MGM that combines standard Full-Function Evaluation (Full-FE) steps and Local-Function Evaluation (Local-FE) steps.\n\nIn a Full-FE step, the model selects a group of target tokens to unmask and compute fresh KV for all tokens, but caching KV only for the non-selected tokens (context or masked but not flagged for unmasking).\nA Full-FE step is followed by a succession of Local-FE steps that progressively unmask the target tokens, recomputing KV only for the target tokens and using the cache for the rest.\n\nThe authors show that, as an inference-only strategy, ReCAP can be applied without retraining to various MGM models, both discrete or continuous.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The paper addresses inference cost, a well-identified bottleneck in generative models.\n- The proposed solution is derived from a common and effective practice in autoregressive models (KV caching), which has been lacking in MGMs. It is simple, does not require retraining, and is easy to integrate.\n- The core hypothesis --stability of KV features across decoding steps-- is well supported by empirical evidence given in the paper. This provides both motivation for the method and interesting insight into MGM behavior.\n- Experiments performed on different MGMs show the general applicability of the approach. The inclusion of continuous-valued models is a welcome addition.\n\nWeaknesses:\n- From a user perspective, deciding how many Full-FE steps (T), how many Local-FE steps (T′), and when to start interleaving them (u) will be critical for balancing speed and quality. The paper provides somewhat limited guidance on how to choose these parameters in practice. Table 1 offers some insight, but only for MaskGIT-r, and the settings there appear somewhat ad hoc (e.g., not always following $u= \\frac{T+T'}{2}$ that seems to be the default for the other experiments). Furthermore, there is no discussion on whether these hyperparameters generalize well across models.\n- The results focus on regimes where ReCAP causes minimal quality degradation. However, it would be interesting to know how the performance evolves in a wider range of values for the repartition between T and T', and when the generation starts to collapse. Currently, the paper leaves these questions open.\n- While a 2–2.5× speedup is certainly valuable, it feels somewhat modest for inference optimization methods.\n\nQUESTIONS:\nMy main question relates to the stability and sensitivity of ReCAP's hyperparameters (T, T′) allocation:\n- How does the efficiency-quality trade-off evolve under different allocations of Full-FE (T) and Local-FE (T′) steps?\nIn particular, what happens when ReCAP operates under more aggressive schedules, with a much higher ratio of Local-FEs to Full-FEs? Is there a clear point where failure modes emerge?\n- While I understand that the authors chose simple, generalizable settings across models, is there substantial headroom left if hyperparameters are finely tuned for each architecture? Also, is the behavior of performance vs. hyperparameter choice consistent across different models, or does it vary widely?\n\nSome more systematic analysis of ReCAP’s hyperparameter sensitivity would both help practitioners estimate deployment effort and offer additional insight into the inner workings of MGMs.\n\nAlso, it's mostly a detail but I'd be curious to know if there a reason why, between two not Local-FE steps, the previously unmasked tokens KV are not added to the cache but appears to be recomputed in Fig3.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-28"}