Title: TTS-VAR: A Test-Time Scaling Framework for Visual Auto-Regressive Generation
Venue: NeurIPS 2025
Accepted: yes
Keywords: Visual Auto-Regressive Modeling

Image Generation

Test-time Scaling
Abstract: Scaling visual generation models is essential for real-world content creation, yet requires substantial training and computational expenses. Alternatively, test-time scaling has garnered growing attention due to resource efficiency and promising performance. In this work, we present the first general test-time scaling framework for visual auto-regressive (VAR) models, TTS-VAR, modeling the generation process as a path searching problem. Inspired by VAR's hierarchical coarse-to-fine multi-scale generation, our framework integrates two key components: (i) At coarse scales, we observe that generated tokens are hard for evaluation, possibly leading to erroneous acceptance of inferior samples or rejection of superior samples. Noticing that the coarse scales contain sufficient structural information, we propose clustering-based diversity search. It preserves structural variety through semantic feature clustering, enabling later selection on samples with higher potential. (ii) In fine scales, resampling-based potential selection prioritizes promising candidates using potential scores, which are defined as reward functions incorporating multi-scale generation history. To dynamically balance computational efficiency with exploration capacity, we further introduce an adaptive descending batch size schedule throughout the causal generation process. Experiments on the powerful VAR model Infinity2B show a notable 8.7% GenEval score improvement (0.69→0.75). Key insights reveal that early-stage structural features effectively influence final quality, and resampling efficacy varies across generation scales.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn the inference stage of VAR, this work employs semantic feature clustering in the early steps to maintain diversity in the sampling process. In the later steps, a reward model is used to compute potential scores and reorder candidate samples, thereby achieving computationally efficient test-time scaling.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**  \n1. This work is the first general test-time scaling framework for VAR models.   \n2. The proposed TTS-VAR achieves effective improvements on the GenEval benchmark.\n\n\n**Weaknesses**\n1. I am concerned about the inference speed of the model. This work starts with a batch size of 8*N and employs additional components such as DINOv2, k-means clustering, and multi-time reward model evaluations. Would these factors significantly reduce the inference speed? If we increase the value of N in the BoN, is it possible to achieve similar performance to TTS-VAR at a comparable inference speed?  \n2. Geneval reflects the semantic consistency of generated images under simple prompts and cannot capture the quality of details in the generated images. The advantages of high-scale resampling do not seem to be effectively captured by Geneval.  \n3. In Tab. 2, higher resampling frequency appears to lead to worse performance. Could the authors provide a more in-depth analysis of this phenomenon?\n\nQUESTIONS:\nAs the first TTS work for VAR architectures, I think it barely meets the acceptance criteria for NeurIPS, although I do have some concerns about this work (see weaknesses).\n\nThe authors adopt N=8 as the final setting, which requires starting inference with a batch size of 64. I am concerned that the performance improvement may be attributed to more sampling rather than the ingenuity of the proposed design.  Inference speed is also a factor that needs to be considered.  \n\nWhat is the performance when using only Clustering-Based Diversity Search during TTS?\n\nLIMITATIONS:\nNO.\nI do not find any potential negative societal impact in this work.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces TTS-VAR, a novel test-time scaling framework for Visual Auto-Regressive (VAR) models, addressing the challenge of enhancing image generation quality without retraining. VAR models generate images hierarchically from coarse to fine scales, but early-stage tokens are hard to evaluate, leading to suboptimal sample selection. TTS-VAR tackles this via:\n1. Clustering-based diversity search\n2. Resampling-based potential selection\n3. Adaptive descending batch size\nEvaluated on the text-to-image VAR model *Infinity*, TTS-VAR improves the GenEval score by 8.7% (0.69→0.75) and outperforms diffusion/AR baselines with 60% fewer parameters.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n\n- **Quality**:\n    - Rigorous experiments across various metrics (GenEval, ImageReward, etc.) demonstrate consistent gains.\n    - Ablation studies validate design choices (e.g., resampling scales, potential scores).\n- **Clarity**:\n    - Well-structured; figures (e.g., Fig 1, 5, 7) effectively illustrate the framework and scale-dependent behaviors.\n    - Limitations and societal impact acknowledged (Sec 6, Appendix).\n- **Significance**:\n    - First general test-time scaling method for VAR models, enabling efficient high-quality generation.\n    - Achieves SOTA results with lower computational costs (e.g., 25% samples vs. Best-of-N).\n- **Originality**:\n    - Novel integration of clustering (for structural diversity) and history-aware resampling tailored to VAR’s hierarchy.\n\n**Weakness:**\n\n- The paper does not provide quantitative or qualitative analysis of how the TTS-VAR framework affects output diversity. While the method emphasizes preserving structural diversity through clustering (Sec 4.2), there is no visualization comparing image variation between TTS-VAR and base Infinity model.\n- The paper inadequately examines the compute-performance tradeoffs inherent in its design choices. Specifically, there is no analysis of how resampling at different scales impacts both computational cost (FLOPs, latency) and final output quality.\n\nQUESTIONS:\n**Questions:**\n\n- How does TTS-VAR framework affect the output distribution’s diversity? In [1], the paper demonstrates the FID and IS after scaling NFEs.\n- What’s the computation-performance tradeoff of different designs?\n- In table 3. and table 4., why does the proposed framework have limited improvements on HPS, CLIP and Aesthetic score?\n- The efficiency of investing compute between diffusion models and VAR models?\n\nReference\n\n[1] Inference-Time Scaling for Diffusion Models beyond Scaling Denoising Steps. Ma, Nanye, et al.\n\nLIMITATIONS:\nThe limitation discussions are sound and clear to me.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents TTS-VAR, a test-time scaling framework designed for visual autoregressive models by formulating generation as a path search task. To maintain computational efficiency, the authors introduce an adaptive descending batch size schedule, which operates similarly to an ensemble method with pruning to achieve a balance between performance and cost. For selecting high-quality candidate images, the method proposes a clustering-based diversity search to preserve structural information at coarse scales, followed by a reward model-based potential resampling to refine details at finer scales. Experimental results demonstrate that TTS-VAR improves generation quality.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths:\n\nThe authors make the first attempt to explore test-time scaling for visual autoregressive models to improve performance, which appears to be an interesting and promising direction.\n\n\n### Weaknesses:\n\n1. **Contribution of adaptive batch sampling:**\n    \n    In Fig. 2, I suggest the authors include additional curves showing memory usage and computational complexity with increasing scale index while using fixed but larger batch sizes (e.g., 2, 4, 8) for comparison. The current figure only illustrates the quadratic growth in computation due to increasing tokens but does not clarify the dependency on batch size. This weakens the motivation for adaptive batch sampling and clustering-based search.\n    \n2. **Comparison with Best-of-N (BoN):**\n    \n    The comparison in Fig. 1 between (b) and (c) seems unfair. BoN operates on a batch size of 2, while TTS-VAR starts with 8 initial samples, followed by clustering and resampling. Intuitively, BoN with resampling (batch size = 8) should perform at least as well as the current method, if not better, since no filtering is required. If a fixed batch size of 8 is feasible, why constrain the approach with decreasing batch sizes? The authors should provide a deeper discussion of this alternative, clarify how their method excels in efficiency, and present quantitative metrics to better evaluate the trade-offs.\n    \n3. **Design of clustering-based diversity search and resampling-based potential selection:**\n    - The clustering approach needs further justification. Why does proximity to a cluster center imply high-quality structure? Additionally, clustering with very few samples (e.g., <8) may not be meaningful. What happens if all candidate samples are of poor quality? More visualizations (e.g., feature comparisons between superior and inferior samples, clustering results of \"diverse\" outputs) would help clarify this.\n    - The reward model is used for resampling, but as noted in Supp. D.2, this overlaps with the model used for quantitative evaluation, potentially biasing the results. Leveraging this prior likely inflates performance metrics unfairly.\n4. **Missing experimental results:**\n    - Ablation of clustering-based diversity search: To validate clustering’s efficacy, the authors should include a baseline where N=8 samples are randomly dropped instead of clustered. The current experiments are insufficient.\n    - While most experiments follow the setting of Infinity, results on the DPG benchmark are missing.\n    - In Supp. Tab. 10, the authors should test λ > 10 to demonstrate that the current parameter choice is optimal.\n5. **Confusing notations:**\n    - L108: Replace i with k.\n    - L158: The subscript of the union symbol should be j, not *bᵢ*.\n    - L137-138: The definition of *N* is unclear and seems inconsistent with the batch size notation in evaluations (e.g., Tab. 1).\n\nQUESTIONS:\n1. I suggest the authors include a detailed table comparing the time/memory cost of the method against each variant and baseline. Since this paper focuses on testing-time scaling behaviors, such a comparison is necessary.\n2. Why use \"clustering\" instead of \"pruning\" or \"selection\"? The choice seems somewhat counter-intuitive.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 1, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe article introduces a novel test-time scaling framework (TTS-VAR) for autoregressive models in the vision domain. The method proposed is able to take any trained text-to-image model and apply multi-scale path search algorithms and use the extra time to improve output quality by making batch sizes adaptive, employing clustering early for diversity and potential-based resampling once images are almost complete. Extensive empirical evidence suggests that TTS-VAR is able boost performance using few fewer parameters without touching the training process.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. Empirical gains are quite solid on GenEval 0.694 → 0.753 at N = 8 and already beats Best-of-N-8 with just N=2 while using 60% fewer parameters than SD-3. \n2. The problem presented is well-motivated since test-time scaling has seen tremendous success in the text domain. The intuition for TTS-VAR makes sense and I think achieving SD-3 level quality without retraining could unlock a wide range of real-world applications with costs significantly reduced. \n3. In general, the methodologies are clear to follow. Motivations for novel techniques such as adaptive batching, potential resampling and clustering are clearly motivated and formulated. \n4. Ablations are comprehensive on the design choices of potential functions (table 3), resampling scales (table 2), clustering scales (table 4) and clustering extractors (table 5).\n\nConcerns:\n1. A problem I have with this paper is the lack of evaluation scope since only infinity-2B is used as the base model and GenEval as the sole benchmarking metric.\n2. Figure 2 only presented Flop and memory information but does not include the type of GPU used and actual GPU hrs used to run the experiments.\n3. There is limited disclosure of hyper parameters used in the studies including but not limited to K (in K-mreans), seeds for experimentation and PCA energy threshold.\n\nQUESTIONS:\n1. Could you provide include more base-models on extended number of benchmark datasets to validate the generation of the proposed framework?\n2. Would it be possible to provide concrete GPU times used on top of flops/memory usage to validate the practical improvements of TTS-VAR? Would be also nice to see how 60% deduction in parameters compared to SD-3 translate to real gains on inference efficiency (memory, time etc).\n3. Could you provide a small set of human feedbacks on how TTS-VAR performs compared to baselines (such as best of n) on top of qualitative benchmarks?\n4. Could you provide detailed hyperparameters used in the studies including but not limited to K (in K-mreans), seeds for experimentation and PCA energy threshold for reproducibility purposes?\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-02"}