Title: Multi-step Visual Reasoning with Visual Tokens Scaling and Verification
Venue: NeurIPS 2025
Accepted: yes
Keywords: MLLMs
Abstract: Multi-modal large language models (MLLMs) have achieved remarkable capabilities by integrating visual perception with language understanding, enabling applications such as image-grounded dialogue, visual question answering, and scientific analysis. However, most MLLMs adopt a static inference paradigm, encoding the entire image into fixed visual tokens upfront, which limits their ability to iteratively refine understanding or adapt to context during inference. This contrasts sharply with human perception, which is dynamic, selective, and feedback-driven.
In this work, we introduce a novel framework for inference-time visual token scaling that enables MLLMs to perform iterative, verifier-guided reasoning over visual content. We formulate the problem as a Markov Decision Process, involving a reasoner that proposes visual actions and a verifier—trained via multi-step Direct Preference Optimization (DPO)—that evaluates these actions and determines when reasoning should terminate. To support this, we present a new dataset, VTS, comprising supervised reasoning trajectories (VTS-SFT) and preference-labeled reasoning comparisons (VTS-DPO).
Our method significantly outperforms existing approaches across diverse visual reasoning benchmarks, offering not only improved accuracy but also more interpretable and grounded reasoning processes. These results demonstrate the promise of dynamic inference mechanisms for enabling fine-grained, context-aware visual reasoning in next-generation MLLMs. Code and datasets are publicly released at https://vts-v.github.io/.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis research addresses the limitations of the static visual encoding paradigm in current Multimodal Large Language Models (MLLMs). It highlights how their pre-processing approach, which relies on fixed visual tokens, hinders dynamic reasoning capabilities and fundamentally differs from the iterative feedback mechanism of human perception. The authors propose a framework: inference-time visual token scaling. This framework models the process using a Markov Decision Process and dynamically adjusts the inference process with a multi-step DPO-trained validator. This allows the model to achieve adaptive, refined analysis of visual content. To validate the effectiveness of their method, the researchers also constructed a specialized dataset called VTS, which includes both supervised trajectories and preference comparisons.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n1. The paper proposes a dataset which may help the training of the reward model.\n\n2. The proposed method is effective for some image-based visual reasoning tasks\n\n\n**Weakness**\n\n1. Figures 1 and 2 are not clear.\n\n2. If the \"reasoning reinforcement polishing\" (likely referring to your iterative refinement process) involves multiple rounds of actions, please elaborate on the key distinctions between your proposed reinforcement learning method and existing multi-round reinforcement learning approaches. Specifically, how does your inference-time visual token scaling mechanism and the use of a multi-step DPO trained validator differentiate it from, or build upon, traditional multi-round reinforcement learning paradigms?\n\n3. Your core motivation is to address the limitations of static visual information input, which leads to incomplete attention during contextual reasoning. While the self-attention mechanism in large language models does allow for varying attention to visual tokens during training and inference, your work introduces a novel approach to dynamic attention adjustment for visual information. It's crucial to acknowledge and explicitly compare your method with prior work that has also tackled dynamic visual attention or iterative perception in MLLMs. Please discuss how your approach, particularly inference-time visual token scaling and the Markov Decision Process modeling, differs from and potentially advances solutions proposed in works such as:\nLMEye: Iterative Perception Network; InstructBLIP; V*\nA clear comparative analysis against these established methods would significantly strengthen your paper's contribution and highlight the novelty of your proposed framework.\n\n4. The experimental results show limited improvement in some datasets, and the improvement may be largely relevant to the sft stage.\n\nQUESTIONS:\nsee the weakness\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper focus on inference-time visual token scaling, via a tool-augmented VLM Reasoner module and a verifier optimized by DPO. The Reasoner can be either API-based models or finetuned open-source models.\nThe proposed framework relies on the self-constructed dataset, a new tool-enhanced CoT-style dataset.\nThis paper also describes engineering details of their data generation pipeline.\nThe method outperforms baselines, both tool-augmented and prompt-based methods, across several visual reasoning benchmarks, demonstrating improved accuracy and tool-using ability. The authors also provide theoretical guarantees that their algorithm ensures a bounded number of reasoning steps, preventing looping or repetitive outputs.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n1. The paper presents a technically solid framework with clear theoretical explanation.\n2. The design and construction of tool-augmented multi-step visual reasoning dataset is well motivated, addressing a notable gap in existing datasets, which would be useful to future research.\n3. The proposed method largely improves the performance of open-source models regarding visual reasoning and grounding tasks.\n\n**Weaknesses:**\n1.Authors claim that their proposed method offers more interpretable reasoning traces. However, no related evaluation or demonstrations of resulting generations of the trained models are provided (authors only provide the example trace in their constructed data).\n  - More discussion about the results from the trained models using their data should be provided to better support the claim of \"more interpretable reasoning traces\". A dataset with longer reasoning traces or traces with diverse tools does not guarantee that the trained models would exhibit exactly desired performance.\n  - Manual evaluation or further comparison is required to support the contribution regarding the term, \"more interpretable reasoning trace\". For example, in Line 92-95, these methods is not **limited** to 1-2 steps, many questions can be well solved only via 1-2 steps.\n2. The verifier is similar to existing feedback-based reflective mechanisms. According to Line 103, the proposed method should differ in both tasks and training datasets. However, the proposed framework is also a tailored dataset whose actions are restricted to tool-use.\n\nQUESTIONS:\n1. Please response to Weaknesses.\n2. From Table2, there could be a conclusion that the improvement main comes from the curated dataset. The gain of the framework itself remains unclear.\n3. Is it possible to add additional experiment of w/o reasoner (as the other ablation study in table3)? This could further demonstrate the effectiveness of the entire framework, otherwise the reasoner could be a dispensable module.\n\nI acknowledge that the dataset could be useful, please provide further explanation and evidence to demonstrate the necessity and effectiveness of the framework (as mentioned above) , so that this paper would not remain significant in dataset engineering only. I will raiss my rating if all questions and concerns are solved.\n\nAdditional suggestion: (NOT weakness) \n1. Some simple visualisations of the framework would assist in understanding of the entire work.\n2. Please carefully check the tables and figure in appendix. They are not properly displayed.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the limitation of conventional multimodal large language models (MLLMs) with fixed and static visual tokens input, hindering iterative refinement of visual understanding. To bridge this gap, the authors propose a framework named Visual Token Scaling with Verification (VTS-V), which enables MLLMs to perform iterative, verifier-guided reasoning over visual content.\n\nVTS-V formulates visual reasoning as a Markov Decision Process (MDP) consisting of two core components: a reasoner that generates visual reasoning steps (planning, action selection, and observation) and a verifier trained via multi-step Direct Preference Optimization (DPO) to evaluate action quality and determine termination. To support training, the authors introduce the VTS dataset, comprising VTS-SFT (315K supervised reasoning trajectories with tool use) and VTS-DPO (301K preference-labeled trajectory pairs).\n\nExperimental results across diverse benchmarks (BLINK, V*Bench, MMStar, MathVista) demonstrate that VTS-V significantly outperforms baselines. For closed-source models like GPT-4o, it improves average accuracy on BLINK from 62.25% to 69.15%. For open-source models (e.g., Qwen2-VL-7B, LLaMA3.2-11B), fine-tuning on VTS-SFT followed by VTS-V yields gains of 4.18% to 7.58%. Ablation studies confirm the verifier’s critical role in filtering suboptimal paths.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n- VTS-V effectively addresses the static inference limitation of MLLMs by introducing dynamic visual token scaling and verifier-guided iteration, aligning more closely with human-like dynamic perception.\n- The VTS dataset (VTS-SFT and VTS-DPO) fills a critical gap in tool-augmented visual reasoning data, providing high-quality supervised and preference-labeled trajectories to train both the reasoner and verifier.\n- Experiments span diverse benchmarks and model types (closed-source and open-source), showing consistent improvements in accuracy and interpretability.\n\n**Weaknesses:**  \n- Although the work is solid, the method is more like a combination of previous works [1] (for iterative observation and reasoning), [2] (for reasoner and verifier separation) and [3] (for tool-use), thus its novelty is limited.\n- The paper highlights \"visual token scaling\" as a key innovation, but it fails to clearly exhibit the scaling mechanism, e.g., a curve demonstrating the scaling law for visual tokens.\n- The verifier’s performance depends heavily on the VTS-DPO dataset, and its ability to adapt to unseen tasks or domains is not thoroughly explored. This raises questions about its robustness in real-world, diverse environments.\n\n[1] Penghao Wu and Saining Xie. $V^*$: Guided visual search as a core mechanism in multimodal llms. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13084–13094, 2024.\n\n[2] ArianHosseini,XingdiYuan,NikolayMalkin,AaronCourville,AlessandroSordoni,and RishabhAgarwal. V-star: Training verifiers for self-taught reasoners. arXiv preprint arXiv:2402.06457, 2024.\n\n[3 ]Yushi Hu, Otilia Stretcu, Chun-Ta Lu, Krishnamurthy Viswanathan, Kenji Hata, Enming Luo, Ranjay Krishna, and Ariel Fuxman. Visual program distillation: Distilling tools and programmatic reasoning into vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9590–9601, 2024.\n\nQUESTIONS:\nSee the Weaknesses section for major concerns. Besides, I would like to ask the following questions:\n\n1. Since the pipeline is modeled as Markov Decision Process, why not train the models use RL algorithms (e.g., GRPO)?\n2. What if we train a single model to perform reasoning actions and verification both, instead of two separate models? Would it be more efficient?\n3. Can we use the trained VTS-V pipeline to construct a new iteration of VTS dataset, and iteratively boosting the framework in a self-teaching manner?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents VTS-V, a novel framework for inference-time visual token scaling in multi-modal large language models (MLLMs). Inspired by the dynamic, feedback-driven nature of human visual perception, the authors model visual reasoning as a Markov Decision Process. The framework consists of a reasoner that plans visual actions and a verifier trained via multi-step Direct Preference Optimization (DPO) to guide the reasoning steps and determine termination. To support this framework, the authors introduce VTS, a new dataset including supervised reasoning trajectories (VTS-SFT) and preference-labeled trajectory pairs (VTS-DPO). They demonstrate consistent improvements over several benchmarks, showing enhanced performance and interpretability.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The iterative inference-time scaling of visual tokens using verifier-guided reasoning presents a new perspective in the field.\n- The iterative nature of VTS-V provides clear, interpretable reasoning paths, enhancing the transparency of model decisions.\n- The new dataset (VTS) provides valuable resources for further research.\n\nWeaknesses:\n- The complexity of implementation and computation inherent in training and inference procedures may limit widespread adoption. I encourage the authors to include a more detailed analysis of inference-time overhead, such as latency, and memory usage across models and datasets. \n- While the authors present comprehensive results for GPT-4o variants on the BLINK benchmark, the evaluation on other datasets such as V*Bench, MMStar, and MathVista is only conducted for open-source models. As GPT-4o represents a strong closed-source baseline, its exclusion from these additional benchmarks limits the understanding of how the proposed VTS-V framework generalizes across diverse reasoning tasks for top-tier models. Including these comparisons would strengthen the empirical claims.\n- The paper observes that integrating VTS-V with GPT-4o yields consistent improvements, even without additional fine-tuning, whereas applying the same method to open-source models (e.g., Qwen2-VL, LLaMA3.2) without fine-tuning leads to significant performance drops. However, this discrepancy is not thoroughly analyzed. The authors are encouraged to investigate and discuss potential reasons behind this behavior—for example, differences in instruction-following ability, pretraining data coverage, or inherent robustness to tool-augmented workflows.\n\nQUESTIONS:\nPlease address the weaknesses above.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}