Title: Robust Cross-modal Alignment Learning for Cross-Scene Spatial Reasoning and Grounding
Venue: NeurIPS 2025
Accepted: yes
Keywords: Cross-modal retrieval

3D vision grounding

benchmark dataset
Abstract: Grounding target objects in 3D environments via natural language is a fundamental capability for autonomous agents to successfully fulfill user requests. Almost all existing works typically assume that the target object lies within a known scene and focus solely on in-scene localization. In practice, however, agents often encounter unknown or previously visited environments and need to search across a large archive of scenes to ground the described object, thereby invalidating this assumption. To address this, we reveal a novel task called Cross-Scene Spatial Reasoning and Grounding (CSSRG), which aims to locate a described object anywhere across an entire collection of 3D scenes rather than predetermined scenes. Due to the difference from existing 3D visual grounding, CSSRG poses two challenges: the prohibitive cost of exhaustively traversing all scenes and more complex cross-modal spatial alignment. To address the challenges, we propose a Cross-Scene 3D Object Reasoning Framework (CoRe), which adopts a matching-then-grounding pipeline to reduce computational overhead. Specifically, CoRe consists of i) a Robust Text-Scene Aligning (RTSA) module that learns global scene representations for robust alignment between object descriptions and the corresponding 3D scenes, enabling efficient retrieval of candidate scenes; and ii) a Tailored Word-Object Associating (TWOA) module that establishes fine-grained alignment between words and target objects to filter out redundant context, supporting precise object-level reasoning and alignment. Additionally, to benchmark CSSRG, we construct a new CrossScene-RETR dataset and evaluation protocol tailored for cross-scene grounding. Extensive experiments across four multimodal datasets demonstrate that CoRe dramatically reduces computational overhead while showing superiority in both scene retrieval and object grounding. Code is available at https://github.com/Yangl1nFeng/CoRe.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nTargeting at locating a described object anywhere across an entire collection of 3D scenes, this paper proposes a Cross-Scene 3D Object Reasoning Framework to realise scene matching and object grounding. This paper also organises a new dataset. Experimental results verify the effectiveness of the proposed RTSA and TWOA modules.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. This paper is generally well-written and easy to follow.\n2. This paper effectively articulates the issues associated with cross-modal alignment, providing a clear and comprehensive explanation from my point.\n\nWeaknesses:\n1. The visual results are not clear. I cannot directly observe the advantages of the proposed method without the aid of the word descriptions, such as Fig. 6.\n2. I think the Eq. (1) can be moved to the end of Section 4 when all the modules have been introduced, which can make the organisation more clear.\n3. More details should be provided, such as the parameter settings of the loss function. \n4. Some texts in the figures are not too small, such as Fig. 5.\n5. A limitation part can be included in the paper to make this research more comprehensive.\n\nQUESTIONS:\n1. The author's method focuses on the accurate alignment of text and objects. Are there any more innovative designs in the perception and components of 3D environments?\n2. Why does the proposed method not work well in the Sr3D dataset? More explanation should be provided.\n3. The authors should give more explanations about the training process of the proposed method.\n4. How about the running speed and model parameters compared with other methods?\n5. For Tab. 1, how to get these statistics？ Are they the statistics provided in the documentation of those datasets or the authors' own rough estimates?\n6. For the targeted masking strategy, how to decide the mask location and probability? Whether the choice of mask have a significant impact on the experimental results?\n\nLIMITATIONS:\nYes. I think this paper has no potential negative societal impact.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a practical and general task called Cross-Scene Spatial Reasoning and Grounding (CSSRG). The authors conduct a thorough analysis of the task and its associated challenges. To address these challenges, they propose a solution framework named CORE, along with a special dataset called CrossScene-RETR. Specifically, CORE consists of two modules that are designed to tackle the respective challenges of CSSRG, mitigating both computational overhead and the risks of cross-modal misalignment. Experiments provide baseline results for CSSRG and demonstrate the superiority of the proposed CORE.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: \n1.This paper introduces a new task and proposes a novel method and dataset for it, along with baseline results and detailed analysis. I believe the overall effort involved is substantial.\n\n2. The paper is well-motivated and easy to follow. The CSSRG task appears to be both more challenging and practical, and it is thoroughly analyzed by the authors.\n\n3. The paper is well-structured, with particularly intuitive and informative figures that aid understanding.\n\n4. The proposed method is shown to be effective through both the methodological design and experimental results. The underlying theoretical framework seems to be solid.\n\n5. The dataset analysis presented in the paper is comprehensive, especially the data construction, statistics, and analysis in the supplementary material.\n\nWeaknesses:\n1.It appears that CORE also needs to iterate through all scenes to match them before grounding objects. Why its efficiency is significantly higher compared to 3D visual grounding methods?\n\n2.Sec. 4 could benefit from a more detailed explanation, as certain aspects remain unclear. For instance, the process by which RTSA obtains common representations is not fully elaborated. Additionally, how does the variation in $q$ help mitigate the impact of false-negatives?\n\n3.The use of data subsets is confusing. It seems there are two different strategies for grouping the data into subsets, yet Fig. 7 analyzes them together. It would be helpful to clarify the essential differences between these grouping methods.\n\nQUESTIONS:\nWhy CORE efficiency is significantly higher compared to 3D visual grounding methods? A more comprehensive descriptions of the method and data subset partitioning should be declared. See the weakness section for more details.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, the authors aim to investigate a novel task named as Cross-Scene Spatial Reasoning and Grounding (CSSRG). Different from the existing 3D Visual Grounding (3DVG) task, CSSRG extend the perception of a described object to an entire collection of 3D scenes rather than the predefined scene in 3DVG. To address this challenge, the authors propose CoRe, which adopts a matching-then-grounding paradigm to effectively reduce computational costs. CoRe comprises a Robust Text-Scene Aligning module (RTSA) for robust scene matching and a Tailored Word-Object Associating module (TWOA) for object grounding. The experiments on various datasets (ScanRefer/Nr3D/Sr3D/RETR) demonstrate the superiority and effectiveness of CoRe compared with existing SOTAs.\n\nSTRENGTHS AND WEAKNESSES:\n**Strength:**\n 1. This paper firstly investigates the Cross-Scene Spatial Reasoning and Grounding (CSSRG) task. Different from many existing works that concentrate on the 3DVG task with predetermined scene configuration, CSSRG can handle an entire collection of 3D scenes. It is more challenging and meaningful.\n 2. The overall organization of this paper is clear and easy to follow.\n 3. The proposed dataset (CrossScene-RETR) is of great significance and inspiring for the future works.\n\n**Weakness:**\n 1. The figures (Fig. 4, 5, 6) are hard to read. The text in these figures are too small.\n 2. I am kind of confused about the “Spatial Reasoning” part in the task of Cross-Scene Spatial Reasoning and Grounding (CSSRG). From Figure 3, I can understand that the proposed pipeline first find the matching scene with the given prompt, and then conduct visual grounding to find the exact spatial location according to the given prompt. However, this process only includes the grounding part, the concept of spatial reasoning seems somewhat overly broad. Maybe more tasks should be involved rather than a single grounding task.\n 3. The introduction of the proposed CrossScene-RETR dataset is too brief in Section 5. I have read the detailed introduction of the proposed dataset in the appendix. I think the proposed dataset is an important contribution to the community and should be highlighted in the manuscript.\n 4. Failure case analysis is missing. The authors are suggested to provide a failure case analysis.\n\nQUESTIONS:\n1. **[Clarification about Spatial Reasoning]**. Could the authors provide some clarification about the “Spatial Reasoning” part in the task of Cross-Scene Spatial Reasoning and Grounding (CSSRG)? As discussed in Weakness #2, I think the proposed method can only handle the grounding task. However the spatial reasoning is a much larger concept, rather than a single grounding task. More tasks should be involved if “Spatial Reasoning” is used in your paper.\n 2. **[Number of Scenes]**. From Figure 2, 141 scenes are used in CSSRG task, whereas only 1 scene is used in 3DVG task. Does the number of scenes affect the performance of CSSRG?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

comments: SUMMARY:
The paper introduces Cross-Scene Spatial Reasoning and Grounding (CSSRG), where an agent must find a language-described object anywhere in a large archive of 3D scenes. To tackle the prohibitive traversal cost and partial text-scene alignment, the authors propose CoRe, a two-stage “match-then-ground” framework combining a Robust Text-Scene Aligning module with a Screening-Attention-based Tailored Word-Object Associating module. They also plan to release the CrossScene-RETR dataset, whose richer descriptions sharpen evaluation, and show CoRe beats thirteen baselines while running ≈250× faster than single-scene methods.

STRENGTHS AND WEAKNESSES:
**Strengths**

* Frames a needed research problem (CSSRG) that reflects real agent memory search.
* Plans to release CrossScene-RETR, enabling future benchmarking.
* Two-stage CoRe cuts computation yet lifts accuracy over 13 baselines.

**Weaknesses**

* Despite gains, CSSRG accuracy stays low (≈23 % Acc\@0.25), so practical use is distant.
* Experiments rely on a single 3D backbone; generality to newer encoders is untested.
* The four 3D VG text datasets use heterogeneous text formats, yet the paper gives no preprocessing strategy for syncing them.
* Scope is limited to indoor static scenes; outdoor or dynamic settings remain unexplored.

QUESTIONS:
I would appreciate if you could address my questions regarding limitations.

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 3
DATE: 2025-07-03