Title: MindJourney: Test-Time Scaling with World Models for Spatial Reasoning
Venue: NeurIPS 2025
Accepted: yes
Keywords: Spatial Reasoning

World Models

Test-time Scaling
Abstract: Spatial reasoning in 3D space is central to human cognition and indispensable for embodied tasks such as navigation and manipulation. However, state-of-the-art vision–language models (VLMs) struggle frequently with tasks as simple as anticipating how a scene will look after an egocentric motion: they perceive 2D images but lack an internal model of 3D dynamics. We therefore propose SpatialNavigator, a test-time scaling framework that grants a VLM with this missing capability by coupling it to a controllable world model based on video diffusion. The VLM iteratively sketches a concise camera trajectory, while the world model synthesizes the corresponding view at each step. The VLM then reasons over this multi-view evidence gathered during the interactive exploration. Without any fine-tuning, our SpatialNavigator achieves an average 7.7\% performance boost on the representative spatial reasoning benchmark SAT, showing that pairing VLMs with world models for test-time scaling offers a simple, plug-and-play route to robust 3D reasoning. Meanwhile, our method also improves upon the test-time inference VLMs trained through reinforcement learning, which demonstrates the potential of our method that utilizes world models for test-time scaling.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes SpatialNavigator, a test-time framework that enhances vision-language models' (VLMs) spatial reasoning by coupling them with a controllable video diffusion world model. Without requiring fine-tuning, the method enables VLMs to iteratively explore 3D scenes by planning camera trajectories while the world model generates corresponding synthetic views; these imagined observations are then evaluated and cached to help answer spatial reasoning questions. Experiments across four VLMs and two world models demonstrate consistent improvements (average +8.1% accuracy on the SAT benchmark), showing that this plug-and-play approach complements—and sometimes surpasses—existing methods like RL-finetuned VLMs, while revealing limitations in long-horizon scene consistency that suggest directions for future world model development.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n\n### 1. Technical Soundness\nThe integration of video diffusion models (for 3D consistency) with VLM-guided search is well-executed. The beam search formulation is reasonable and addresses combinatorial explosion in trajectory space. Pipeline (Fig. 2) and algorithm (Alg. 1) are clearly explained. Key components (world model formulation, action space, VLM scoring) are detailed in separate subsections.\n\n### 2. Practical Impact\nAddresses a critical limitation of VLMs (static 2D reasoning) for embodied reasoning. The plug-and-play nature makes it accessible for real-world deployment.\n\n### 3. Rigorous Evaluation\nExtensive experiments on the SAT benchmark (synthetic + real images) with multiple VLMs (GPT-4o, o1, etc.) and world models (SWM, SVC), demonstrating consistent gains (+8.1% avg.). Ablation studies validate design choices.\n\n---\n\n## Weaknesses\n\n### 1. Clarify Technical Novelty  \nThe proposed use of a video world model for test-time scaling in spatial reasoning appears incremental, as conditioning video diffusion on camera poses builds directly on prior works (e.g., CamCtrl, Stable-Virtual-Camera). The core novelty seems to lie in the *interaction* between the VLM and the world model rather than the individual components. The authors should explicitly highlight and differentiate their key technical contributions to avoid misperception of triviality.  \n\n### 2. Address World Model Limitations  \n- **Long-horizon consistency**: Breakdowns in multi-step reasoning (e.g., Fig. 4a) suggest the approach is constrained by current video diffusion models’ limitations. A deeper analysis of failure modes would strengthen the discussion.  \n- **Quantitative evaluation**: Metrics like FVD or LPIPS for view synthesis quality are absent. Including such analyses would provide a more rigorous assessment of the world model’s capabilities.  \n\n### 3. Analyze Computational Trade-offs  \nThe test-time search requires multiple world model calls, but the computational costs (latency, throughput, or energy usage) are not discussed. A brief analysis of scalability—especially for real-world applications—would help readers understand practical limitations.  \n\n### 4. Expand Evaluation Scope  \nThe focus on the synthetic SAT benchmark limits the generality of claims. Evaluating on additional spatial reasoning tasks—such as real-world 3D scene understanding (e.g., [1]) or multi-view consistency (e.g., [2])—would better demonstrate the method’s robustness and broader applicability.  \n\n### 5. Include Missing Baselines  \nComparisons with alternative approaches are needed to contextualize the benefits of test-time scaling:  \n- **3D-aware VLMs**: How does the method compare to architectures explicitly designed for spatial reasoning?  \n- **End-to-end fine-tuning**: Does test-time adaptation outperform direct model adjustments?  \n\n\n[1] Yang, J., Yang, S., Gupta, A.W., Han, R., Fei-Fei, L. and Xie, S., 2025. Thinking in space: How multimodal large language models see, remember, and recall spaces. In Proceedings of the Computer Vision and Pattern Recognition Conference (pp. 10632-10643).\n\n[2] Yeh, C.H., Wang, C., Tong, S., Cheng, T.Y., Wang, R., Chu, T., Zhai, Y., Chen, Y., Gao, S. and Ma, Y., 2025. Seeing from another perspective: Evaluating multi-view understanding in mllms. arXiv preprint arXiv:2504.15280.\n\nQUESTIONS:\n1. **World Model Quality**: What are the FVD/LPIPS scores for generated views at 1/3/5 steps? Include metrics for both indoor and outdoor scenes.\n\n2. **Failure Analysis**: What are the top 3 failure modes of your world model (e.g., object disappearance)? What percentage of errors do they account for?\n\n3. **Compute Efficiency**: What is the end-to-end latency for SAT questions (breakdown: world model calls vs VLM scoring)? How does it scale with beam width B?\n\n4. **Real-World Generalization**: Can you show results on real embodied tasks (ScanNet in Thinking in Space)?\n\n5. What is the human performance gap on SAT after your improvements? What errors remain?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-22"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a novel spatially-aware beam search algorithm that allows to improve the performance of existing VLMs at test-time by coupling a VLM with a posed-controlled diffusion models that acts as a world model.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n* The idea of “adding” a visual/spatial dimension to beam search is thought-provoking. One can imagine extending the same principle to other search algorithms.\n* Experiments are fairly comprehensive: the authors evaluate on both proprietary  and open-source  VLMs, across synthetic and real SAT benchmarks, and with two distinct world models (their SWM and SVC). I particularly appreciate showing SVC even tho the results on o1 are not as good.\n* Reported gains are consistent.\n\n\nWeaknesses:\n* Inference cost. Even vanilla beam search is slow; adding a video-diffusion rollout at every node seems expensive for simple VQA tasks. The paper gives no wall-clock numbers or cost–accuracy plots, so practicality remains unclear.\n* Single-view assumption. The pipeline assumes exactly one reference image; the authors list this as a limitation themselves . It is unclear how SBS would handle multiple views or a short video.\n* Reliance on world-model fidelity. The approach assumes the pose-conditioned diffusion model can render a faithful scene. In practice, a single image underspecifies geometry, so hallucinations could hurt more than help. The authors also acknowledge this,.\n* Scalability knobs left unexplored. Apart from the brief ablation, the paper mostly fixes one search configuration for all experiments . It would be useful to see accuracy vs. (a) action-set size and (b) beam depth to understand the compute/accuracy trade-off. For instance, what’s the performance if the action space is reduced to only one action vs two, and the depth of the tree is increased?\n\n\nOverall, the idea is interesting and the empirical gains seem solid, but the paper needs more discussion (and data) on computational cost and on when the world-model may fail.\n\nQUESTIONS:\n1. Table 1 (SAT-Real).\n    * Why are GPT-4.1 + SN(SVC) and InternVL + SN(SVC) missing?\n\n\n2. World-model quality.\n    * Can you quantify how the DM quality correlates with downstream accuracy? (An experiment where you degrade SWM (e.g. an early checkpoint) or compare longer roll-outs would help.)\n    * Could you provide a simple heuristic or automatic self-check so practitioners know when their world-model is “good enough” for the presented methodology?\n\n\n3. Runtime.\n    * Please report the average wall-clock time per VQA  and compare it to the vanilla VLM baseline.\n    * A plot of accuracy vs. inference-time budget would let readers judge whether the reported  gain justifies the extra compute.\n\n4. Hyper-parameter sensitivity.\n    * How robust is the algorithm  to the thresholds  ?\n    * What happens if you restrict the action space to {move-forward} only but allow deeper search, or conversely allow all three actions but only one step?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-30"}

comments: SUMMARY:
Summary
This paper introduces SpatialNavigator, a test-time scaling framework designed to enhance the spatial reasoning capabilities of vision-language models (VLMs) by pairing them with action-driven, pose-conditioned video world models. The pipeline operates by letting the VLM iteratively plan exploratory camera trajectories, which the world model renders into imagined observations. These are then used by the VLM to answer spatial-reasoning queries, leveraging multi-view synthetic evidence at inference time without any additional training.

STRENGTHS AND WEAKNESSES:
Strengths And Weaknesses

Strengths:
1. The paper identifies a core challenge in spatial reasoning for VLMs, emphasizing the inability of current models to mentally simulate 3D scene dynamics from 2D inputs.

2. The strategy of coupling an off-the-shelf VLM with a video world model is systematic and well grounded.

3. This paper is well-written and well-organized.

Weaknesses:
1. The performance of the entire system appears to heavily depend on the quality of the world model.

2. The test-time computation required for multiple simulated rollouts, per-step VLM scoring, and evidence buffer construction could be substantial, especially for deeper search settings and larger action spaces

3. Although the method is compared to some RL-based TTS VLMs, more explicit analysis or comparison to recent test-time ensembling, chain-of-thought reranking, verifier-based TTS, or model-based counterfactual data augmentation could provide a clearer picture of strengths relative to the broader TTS design space.

QUESTIONS:
See weakness

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 2
IMPACT: 2
ORIGINALITY: 3
DATE: 2025-07-03

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces SpatialNavigator, which frame a Vision reasoning task with a test-time scaling framework. This framework significantly enhances the 3D spatial reasoning capabilities of many different Vision-Language Models (VLMs). The core idea is to address the inherent limitation of VLMs in understanding 3D dynamics from static 2D images by simulating a exploration of a 3D environment. \n\nSpatialNavigator achieves this by using a VLM with a controllable video world model: the VLM iteratively proposes camera trajectories, and the world model synthesizes the corresponding egocentric views. The VLM then leverages the gathered evidences to perform spatial reasoning.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n- The method presents a highly intuitive approach to spatial reasoning, effectively mimicking how humans might mentally simulate movement to understand a scene from different perspectives. Starting from a static image and building a dynamic 3D understanding through simulation.\n- A strength of SpatialNavigator is its design as a test-time compute framework. This makes it inherently model-agnostic. Results show that the method generally improve performance of several baseline VLM models, including strong commercial ones.\n- It is particularly interesting to observe that the information provided by the world model appears orthogonal to the knowledge learned by RL-trained VLMs (e.g., \"O1 style\" models). This suggests that the world model provides a unique and complementary source of information\n\nWeaknesses:\n - As a test-time compute framework, the iterative Spatial Beam Search, which involves generating multiple hypothetical trajectories and synthesizing corresponding views, incurs significant computational cost at inference time.  While this is understandable, this cost should be discussed more thoroughly, and hopefully with suggested future directions on how to improve this.\n\nQUESTIONS:\n- The current framework's effectiveness is demonstrated with a relatively small search depth. Beyond the computational cost, are there other fundamental considerations or architectural limitations that currently constrain scaling the Spatial Beam Search to significantly greater depths, and what are your ideas for overcoming these to enable more complex, long-horizon spatial reasoning?\n- Considering the observed sensitivity of the world model's fidelity to out-of-distribution scenes and extended trajectories, what specific advancements in world model training or architecture do you believe are most critical?\n\nLIMITATIONS:\n- The framework currently operates within a relatively small, predefined set of primitive egocentric actions. While sufficient for the tasks presented, this limited action space raises questions about its direct generalizability to more complex, nuanced, or interactive real-world scenarios", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-04"}