On the Generalization Capacities of MLLMs for Spatial Intelligence
Authors: Gongjie Zhang
Wenhao Li
Quanhao Qian
Jiuniu Wang
Deli Zhao
Shijian Lu
Ran Xu
Venue: ICLR 2026 Oral (2026)
Keywords: 3D Computer Vision
Multimodal Large Language Model
Spatial Intelligence
Embodied AI
Primary area: foundation or frontier models, including LLMs
TL;DR: We show that RGB-only MLLMs are fundamentally flawed for spatial reasoning due to an inherent geometric ambiguity, and propose a camera-aware MLLM framework that incorporates camera intrinsics for robust, generalizable spatial intelligence.

Abstract: Multimodal Large Language Models (MLLMs) that directly process RGB inputs for tasks like 3D localization and navigation have shown remarkable potential. However, we argue that these ``RGB-only'' approaches are fundamentally flawed in their ability to generalize across cameras. By ignoring camera parameters, they entangle an object's physical properties with the camera's perspective, creating an irresolvable ambiguity. We show this leads MLLMs to overfit to the training camera distribution, rather than learning true and generalizable 3D geometric principles. To address this, we propose Camera-Aware MLLM framework for spatial MLLMs. It learns generalizable spatial reasoning by: (i) injecting camera intrinsics via a dense embedding that conditions each visual token; (ii) introducing a camera-aware data augmentation strategy that synthetically varies camera parameters, forcing the model to disentangle camera properties from scene content; and (iii) distilling geometric priors from a 3D vision foundation model. Extensive experiments demonstrate that camera-aware MLLMs substantially outperform their naive counterparts, particularly in cross-camera generalization tests on spatially-grounded tasks, indicating that camera-awareness is not only beneficial but also a prerequisite for robust and generalizable spatial intelligence in MLLMs.

Published as a conference paper at ICLR 2026
ON THE GENERALIZATION CAPACITIES OF MLLMS FOR SPATIAL INTELLIGENCE
Gongjie Zhang 122 Wenhao Lii
Quanhao Qian'²
Jiuniu Wang
Deli Zhao
Shijian Lu Ran Xu
'DAMO Academy, Alibaha Group
HuPan Lab
'Nanyang Technological University
P Project Page: https://githin httpss///gg.ihaoooo/rrreeeeeeeee
ABSTRACT
Multimodal Large Language Models (MILMs) that directly process RGB inputs for tasks like 3D localization and navigation have shown remarkable potential. However, we argue that these "RGB-only" approaches are fundamentally flawed in their ability to generalize across cameras. By ignoring camera parameters, they entangle an object's physical properties with the camera's 's perspective, creating an irresolvable ambiguity. We show this leads MLLMs to overfit to the training camera distribution, rather than learning true and generalizable 3D geometric prin- ciples. To address this, we propose Cameru-Aware MLLM framework for spatial MI..Ms. It learns generalizable spatial reasoning by: (i) injecting camera intrin- sics via dense embedding that conditions each visual token; (ii) introducing a camera-aware data augmentation strategy that synthetically varies camera param- eters, forcing the model to disentangle camera properties from scene content; and (iii) distilling geometric priors from a 3D vision foundation model. Extensive ex- periments demonstrate that camera-aware MLLMs substantially outperform their naive counterparts, particularly in cross-camera generalization tests on spatially- grounded tasks, indicating that prerequisite for robust and generalizable spatial intelligence in MLLMs.
is not only beneficial but also a
INTRODUCTION
Multimodal Large Language Models (MLLMs) are rapidly advancing the frontier of spatial intelli- gence, enabling AI that can perceive and reason about 3D environments through natural language. While early approaches often relied on explicit 3D representations such as point clouds (Xu et al.) 2024 Hong al.) 2023 Wang et al., 2024). a prominent paradigm has emerged: feeding RGB images or videos directly into MLLMs for end-to-end training on spatial tasks like 3D) localiza- tion, depth estimation, relational understanding. and navigation (Zheng et al.] (025) Zhang et al.) 2025 Chen al., 2024a). This RGB-centric methodology, not relying any 3D data, has shown impressive results, suggesting that MLLMs can implicitly learn spatial principles from 2D data.
However, we identify fundamental flaw in this paradigm: the omission of camera intrinsic pa- rameters. This oversight creates an irresolvable geometric ambiguity that undermines cross-camera generalization. As illustrated in Fig. D in the pinhole camera model, a fronto- contt-paraallel object of physical height H at depth Z projects to an image height
hproj f H
(1)
Z
Equation iindues an equivalence class of image observations: for any À >>> 0.
H, (Af. H, AZ) ~ (f.AH,
(2)
all yield the same hproj- Thus, without camera intrinsics, an RGB-only MLLM cannot disambiguate a nearby small object from a distant large one, nor can it separate depth changes from focal-length (zoom) changes. The ambiguity exacerbated by principal-point shifts and pixel aspect ratio: even when two images "look" similar, their per- per-pixel rays differ if camera intrinsics differ, leading models learn camera-specific shortcuts instead of generalizable 3D principles. Prior work in monocular metric depth estimation (Yin et al. 2023) Piccinelli et al.) (024) has shown that canonicalizing intrinsics or injecting them explicitly is crucial for cross-camera generalization; we believe that the same lesson must carry over to ILLM-based spatial reasoning.
"qqual Contribution.
Corresponding Author.

---

Published as a conference paper at ICLR 2026
A: Without carrero intrindics, I connet fell the evect 3D iecatiare even with exect dbjectil sizall
To bridge this gap, we in- troduce the Camera-Aware MLLM framework, designed to make spatial reasoning explicitly camera-aware via three core technical inno- vations. First, we develop a dense camera embedding mechanism that conditions every visual token the corresponding camera's ' ray direction derived from in- trinsic parameters, enabling the model to reason about the geometric relationship between pixels and 3D space. Second, recognizing that existing 3D datasets often lack sufficient camera diversity as com- pared with 2D datasets, propose a camera-aware data augmentation strategy. By synthetically varying camera intrinsics and applying the corresponding geometric transformations to the visual input, we compel the model to disentangle camera properties from scene content. Third, further ground our model in robust geometric principles, we leverage pre-trained 3D vision foundation model, trained on millions RGB-depth pairs across diverse cameras, to distill geometric priors, enriching the MILM'ss understanding while maintaining an efficient, EGBBonly inference pipeline.
1== 2/
bg by
What's the location of the girette
Figure 1: Illustration of the inherent geometric ambiguity in RGB- only spatial reasoning. The same 2D image can result from different 3D scenes: nearby object with a wide-angle lens can appear iden- tical to a distant object with a telephoto lens. This ambiguity makes generalizable 3D localization from single RGB image ill ill-posed problem, especially when camera intrinsics are unknown.
To validate our framework, we conduct extensive experiments on the cross-camera generalization capability of spatial MLLMs. Our experiments reveal stark performance gap: camera-agnostic baselines fail catastrophically on out-of-distribution cameras, whereas our camera-aware models maintain robust performance. These findings demonstrate that explicit camera awareness is crucial for attaining reliable spatial intelligence in MLLMs.
In summary, our contributions are threefold. First, we provide an in-depth analysis revealing the inherent geometric ambiguity in RGB-only spatial reasoning, both theoretically and empirically, and demonstrate that without camera intrinsics, MLLMs cannot learn true and generalizable 3D geomet- ric principles. Second, we propose the Camera-Aware MLLM framework, the first architecture to explicitly address geometric ambiguity in spatial reasoning through dense camera embeddings, ge- ometric prior distillation, and camera-aware augmentation. Third, extensive experiments verify the effectiveness of our method, establishing camera-awareness as prerequisite for generalization and offering clear blueprint for future research. Our work argues for a shift from merely processing pixels to understanding the geometric principles governing their formation, steering the field toward truly generalizable spatial AI.
RELATED WORK
Multimodal Large Language Models (MLLMs). MLLMS extend the powerful reasoning and
language capabilities of LLMs to visual modalities like images (Li et al.
2023 Alayrac al. 2022
2022] Liu et al. 2023; 2024) and videos (Zhang et al. 22023] Lin et al. 2023). By aligning visual encoders with large language models, they excel di diverse range of vision- lision-language tasks, includ- ing object grounding (Peng et al.) 2023, Zhang et al. 2024a), image captioning (Lee et al. [024] Hua et al.) 2025). visual question answering (Jang al. 2202h Kuang et al.) 2025), and complex reasoning (Chen et al. 2024b Huang et al. 2025). Current research continues to scale models, expand their capabilities
video and fine-grained understanding, and develop more sophisticated
architectures (Hurst et al. 2024 Comanici et al. 2025 Bai et al. 2025), establishing MLLMs as foundational technology in AL.
MLLMS for Spatial Intelligence. The impressive capabilities of MLLMs have been naturally extended to spatial intelligence for understanding 3D environments. One line of work directly pro- cesses 3D representations, such as point clouds, for tasks like 3D question answering and localiza- tion (Hong et al. 2023 Xu et al. 2024 Wang et al.) 2024 Chen et al. 2024c: Guo et al. 2025 Miao et al. (025). However, the relative scarcity of large-scale 3D data has motivated an alterna- tive, RGBB-nly paradigm. In this approach, standard 2D MILLMs are trained implicitly grasp
2

---

Published as a conference paper at ICLR 2026
Table 1: Generalization failure of camera-agnostic MLLMs. 3D object detection performance drops when trained mixed data sources evaluated on resized images, exposing fundamental lack of robustness and generalization.
common classes
Model
Train Data
Evaluate Data
Po.25
Ro.25
Flo Fl..25
ScanNet
ScanNet-vall
47.5
44.2
45.7
Qwen2.5-VT.
ARKitSoenes 3RScan Matterport3D SUN-RGRD Objectron
33.3
35.4
ScanNet
ScanNett-val
26.4
24.3
ScanNet
ScanNett-val x1.2
33.0
30.5
31,6
ScanNet
ScanNet-val
48.3
45.0
46.5
VG-LLM
ScanNet ARKitScenes 3RScan Matlerpont3D SUN-RGBD Objectron
ScanNet-vall
49.5
43.6
46.0
ScanNer
ScanNet-vval x0.8
26.7
25.0
25.8
ScanNet
'ScanNet-val x0.8' refers ScanNet validation sot images being resized by factor 0.8.
spatial concepts directly from images or videos, offering greater data scalability and showing signif-
icant promise (Chen al. 2024a Zhang et al.] 2025 Zheng et al., 2025). Vision-Language-AAe Action
(VLA) models (Zitkovich et al. 2023 Kim et al. 2024 Jiang et al., 2025a Qian et al., 2025) for robotics and autonomous driving further highlight the potential of MLLMs to generate real-world actions from visual input. Our work operates within this RGB-only paradigm but identifies and re- solves critical oversight: the systemic neglect of camera intrinsic parameters, which fundamentally limits cross-camera generalization capabilities.
Monocular Metric Depth Estimation (MMDE). MMDE, which predicts per-pixel metric depth from single RGB image. is classic ill-posed problem due to the inherent scale ambiguity of 2D
projection. While early methods (Eienn et al. 2014; Patil al. 2022] Bhat et al. 2021 Piccinelli
et al. 2023) failed generalize across cameras, breakthrough works like Metric3D (Yin et al. 2023]
Hu et al. (2024) and UniDepth (Piceinelli et al. 2024 2025) achieved cross-camera generalization
by explicitly considering camera intrinsics. Their key strategies involve either canonicalizing inputs with known intrinsics or conditioning the network on intrinsics, thereby disentangling camera ge- prerequisite for robust generalization. Our work draws inspiration from this insight, extending the principle from the specialized task of depth estimation to the broader domain of spatial reasoning MLLMs and leveraging universal MMDE model (Piccinelli et al. 2025) to distill geometric priors.
ometry from scene content. These works have established that camera-awareness
BRITTLENESS OF
MLLMS IN SPATIAL REASONING
3.1 THE CHILLENGG OF SPATIALLY-GROUNDEE TASKS
We categorize spatial reasoning tasks into two types: (i) Relational tasks involve qualitative spatial relationships (e.g., "Is the cup to the left of the monitor?") and do not require precise measurements; (ti) Spatially-
tasks de- mand quantitative 3D understanding with queries or answers anchored
coordinate frame (e.g.
"Where is the 3D location of the red chair?" or "Describe the object at (x.yzz).").
tasks is crucial for embodied AI in applications like roboties and autonomous driving. Yet, even advanced mod- els like GPT-40 (Hurst et al. 2024) and Gemini- 2.5 (Comanici et al. 2025) struggle with reli- able 3D grounding. Our work aims to bridge this critical gap, moving MLLMs beyond qualitative recognition towards the quantitative precision re- quired for true spatial intelligence.
Mastering
D.7
Figure 2: ure in a camera-agnostic MLLM (finetuned Owen2.5.VI.). Simply resizing the input image during inference induces a systematic shift in 3D localization.
Visualization of generalization fail-
3

---

Published as a conference paper at ICLR 2026
3.2 EMPIRICAL EVIDENCE OF GENERALIZATION FAILURE
reasoning remains largely an open problem, pioneering works have shown promising results. Notably, VG-LLM (Zheng et al. [225] has demonstrated that MLLMs can be trained/finetueed to reason about 3D space from video data, and generalist MLLMs like Qwen2.5-VL (Bai et al. 2025) can achieve decent results after targeted fine-tuning. In our initial experiments, we confirmed their effectiveness by training and evaluating VG-LLMM (Zeng et al,) 2025) and Qwen2.5-VL (Bai et al. 2025) for single-fraum frame 3D object detection the most basic spatially-roondder task-on on the ScanNet dataset (Dai et al. 2017), establishing a strong single- dataset baseline shown in Table [ However, this success proves be superficial when these models are pushed to generalize.
While the challenge of
Mixed-Source Datasets. The strength of MLLMs lies in their ability to scale up with diverse and large-scale data. However, our attempts to leverage this strength for spatial reasoning led to a counterintuitive outcome. As shown in [ when train baseline models on a large, aggregated datasets composed of multiple indoor scene collections, their per- formance the ScanNet validation set drops. This suggests that the models confused by the conflicting geometric signals from different camera sources, failing to generalize.
Failure of Scaling-Up
Failure under Simple Geometric Transforms. To isolate the cause of this fragility, ducted a controlled experiment. We trained a model exclusively on ScanNet and tested it on the same dataset's ss images after applying simple resize-and-pad (or resize-and-crop) transformation-a common preprocessing step. As shown in Table D performance collapsed under this simple trans- formation. This is not a minor degradation simply caused by resolution change; visualizations of the output (e.g.. Fig. 2 reveal that the predicted 3D locations become systematically and severely offset. This finding strongly suggests that the model has not learned generalizable geometric principles. In- stead, it has severely overfit to the specific resolution of the training images, a property that proves to be brittle and non-generalizable. While Table [ establishes the baseline failure on cross-camera generalization, in our experiments (Fig 6 and T),le5), we show that the proposed Camera-Aware MLLM framework greatly mitigates this failure in the more challenging cross-cament setting.
3.3 ANALYSIS: THE UNRESOLVED 3D GEOMETRIC AMBIGUITY
The empirical failures observed in Section 3.2 trace back to the geometric ambiguity inherent in any camera-agnostic approach. Without knowledge of camera intrinsics, a model cannot disen- tangle scene properties from camera properties, leading over- fitting on sensor geometry rather than learning true 3D principles.
Trequsas
Theoretical Analysis: A Problem of Indistinguishabiity, The relationship between 3D world and its 2D projection is for- mally described by the pinhole camera model. As established in Eq. [ the projected height of fronto-parall object, Aprojj is function of its physical height H. its depth Z, and the camera's 's vertical focal length fy: hrroj Hy /Z. This equation gives rise to an equivalence class of scenes that are indistinguishable from single RGB image. We can formalize at least critical types of ambiguity:
Camera intrinsics distribttion for Owen2.5-VL. after preprocessing.
Focal-Depth Ambiguity: A change in focal length is observa- tionally equivalent to a change in depth. For any scaling factor A > O, the configuration (Afys H, AZ) yields the same pro- jected height as (fy, H, 2). A camera zooming in is indistin- guishable from an object moving closer.
Size-Depth Ambiguity: An object's physical size is con- founded with its depth. The configuration XH, XZ) ( is in- distinguishable from (Sy, H, Z). A small, nearby object can project to the same size as a large, distant one.
1. Calaa
Carneru intrinsics distribution VG-LLM after preprocessing
Figure 3: Multi-modal distri- bution of camera intrinsics in mixed datasets.
While MLLMs can partially mitigate the size-depth ambiguity by leveraging strong priors about object sizes (e.g., knowing a "chair" has typical height H), this mechanism is critically de-
4

---

Published as a conference paper at ICLR 2026
pendent a stable, known focal length. The focal-depth ambiguity fundamentally undermines this prior-based reasoning. A camera-agnostic model forced to assume a canonical focal length, Jy,assmedd implicitly learned from its training distribution. Any deviation in the test camera's fy from this assumption will systematically corrupt its depth and scale estimations. A formal derivation of these ambiguities is provided in Appendix A
Explaining the Failures. The above theory provides an explanation for our empirical findings.
1. Failure on Mixed-Souree Datasets. The performance degradation on aggregated datasets is direct consequence of the model being exposed to multi- distribution of camera intrinsics. As illustrated in Fig. [3] datasets like ScanNet (Dai et al. (017). ARKitScenes (Baruch al. 2021), 3RScan (Wald et al. 2019) and Matterport3D (Chang et (017)) each possess distinct and clustered focal length distributions. camera-agnostic MLLM, attempting to learn single fy fygassuned, faced with conflicting geometric signals. It cannot converge to coherent model, resulting in confusion and a net decrease in performance, as averages over incompatible geometric worlds.
2. Failure under Geometric Transforms (Image Resizing). The resizing experiment pro- vides the most direct proof of this flaw. Image resizing not mere change in resolution; is explicit transformation of the camera's intrinsic parameters. Resizing image by fac- tor s mathematically equivalent to scaling the focal lengths and the principal point coordinates: (fz: fy, Cz: Cy) (s fz,8 fy. Cr, 8 Cy). A model trained learned to operate with fymmin When presented with a resized image, its internal reasoning becomes:
niginal-resolutinn images has
f y.ssuueed H priir
fy.uuin H pricr
Z physiccl
Zpeed
Upproj.esizeed resized
fs Jrann pphysical / /physical
This predicts a systematic error: the model's depth estimate will be inversely proportional to the resize factor s. This explains the huge performance drop in Table ! and the systematic localization offset as observed in Fig. 2
In summary, the failure of camera-agnostic MILMS not limitation of model scale or archi- tecture, but stems from fundamental information deficit: the absence of camera intrinsics. Therefore, for MLLMs achieve robust and generalizable spatial intelligence, they must be made explicitly camera-aware.
4 CAMERA-AWARE MLLM FRAMEWORK
We now present our proposed Camera-Aware MLLM Framework, designed to explicitly resolve the inherent geometric ambiguity of RGB-only MILMs for spatial reasoning. A naive solution might be to adapt canonicalization strategies from Metric3D (Yin et al. (002)), which resample all images shared virtual camera. However, this impractical for MLLMs, it is both computationally prohibitive (causing lots of invalid visual tokens) and relies on precise camera intrinsics that are rarely available for large-scale datasets. Therefore, our framework pursues more scalable and flexible strategy: instead of transforming the visual data, we condition them directly on camera parameters and geometric priors.
4.1 ARCHITECTURE OVERVIEW
As shown in Fig. 4 (a), the model processes text inputs through a standard text encoder. For visual inputs (images videos), each frame is encoded by Geometry-Aware Visual Encoder, which en- riches visual tokens with both raw appearance and geometric context derived from camera intrinsics.
These tokens are then projected into the LLM, following the standard MLLM paradigm (Liu et al.) 2023 Bai et al.) 2025) of joint multimodal reasoning. The core design lies in Geometry-Aware Visual Encoder (illustrated in Fig. (b)), where encode camera intrinsics and geometric priors into visual tokens, described next.
4.2 CAMERA RAY EMBEDDING
Consistent with standard MILM architectures, an input image is first processed by 2D visual encoder (e.g.& a Vision Transformer) to produce a grid of visual tokens, denoted as Fvis E B Hx DD
5

---

Published as a conference paper at ICLR 2026
While these tokens capture rich mantic and appearance information, they are geometrically ambiguous, lacking inherent knowledge of their position within the camera's viewing frustum.
(a)
Asswrr: The 30 bestice of the gratfe Dcpenter s_perher A_semer /LARS, 1_8X4, 1_F2 pnch,
Cwaera-Aware Multimodal Large Language Model
GAVE
GAVE
GAVE
GAVE
To resolve this ambiguity, we intro- duce a Dense Camera Ray Embed- ding that explicitly conditions each visual token on its corresponding line-of-sight, derived from the cam- era's intrinsics. With given intrin- sics (fa- Jy: Cu. Cy), we compute the normalized direction components for each token grid position (i,j), which corresponds to image coor- dinate (Meg, vi)): R2 [i. uzj-c and Ry[i.j]
(b)
Camera Embedding
2D Visual Encoder
3D Geometric Prior Distilaaoor
1x
"y-"y We also include the global focal length val- ues, fa and fv. as part of the em- bedding for every token. They are then encoded using sinusoidal em- bedding layer (Vaswani et al.) 2017), generating dense camera embed- ding Eeam E #xx WDD, which is then fused with the visual features Fvis via element-wise addition. This straightforward mechanism ensures that each visual token is not just descriptor of local semanties but is also grounded in its geometric context, making explicitly aware of its specific line-of-sight into the 3D world.
MMDE
Cloud
Figure 4: The proposed Camera MLLM Framework. (a) The overview the architecture, where (b) Geometry- Aware Visual Encoder (GAVE) injects camera-awareness and 3D geometrie priors into the MI..M.
4.3 CAMERA-AWARE GEOMETRIC AUGMENTATION
Even with explicit intrinsic conditioning, effective learning requires exposure to diverse cameras. However, most 3D datasets are cap- tured with limited sensor setups, resulting in narrow distributions of intrinsics. To address this limitation, we propose a Camera-Aware Geometric Augmentation strategy, shown in Fig. During train- ing, we synthetically perturb intrinsics by:
Scaling: resizing the image by factor s, updating intrinsics (Sz, Ca, Gy) - (8f2,, 8fy, SCa, scy); Shifting:
Update
translating the principal point (cz, Cy) to simulate off- center projections;
Importantly, both the image and its intrinsics are updated consis- tently, ensuring geometric correctness. This forces the model to disentangle scene content from camera geometry, equipping with robustness against cross-camera distribution shifts.
Crop
Update
4.4 GEOMETRIC PRIOR DISTILLATION
While direct 3D annotations for MLLM training are scarce, large- scale RGB-depth pairs are abundant. To leverage this resource, we distill rich geometric priors from state-of-the--at Monocular Metric Depth Estimation (MMDE) model, UniDepth v2 (Piccinelli ct al.) 2025), which was pre-trained on over 10M RGB-depth pairs across diverse cameras.
Figure Illustratoon of camera-aware geometric aug- mentation.

---

Published as a conference paper at ICLR 2026
(4) Single-Frame 3D Detection @ ScanNot
(b) Video 3D Detection @ Sean
(c) Vidoo Visual Grounding & ScanRefor
68.7
g
F18R35 (10 Caomio Clasce)
Filtias (3) Communt Clance)
x
Oveils
VGLL15-B
02044
03
21
13
Reecals
Facier
Figure 6: Cross-camera generalization on spatially-groundn tasks. While cameer--ggnsttcc MLLMs (Qwen2.5-VL, VG-LLM) fail catastrophically on altered camera geometries by rescaling, our method maintains robust performance, proving its ability to generalize across cameras.
For each training image, we use the frozen UniDepth v2 model to predict a dense 3D point cloud, which is then embedded into a geometric prior embedding Egco E lRll xD Egoo is also added to Fvis, further enriching them with explicit 3D structural information.
Notably, geometric prior distillation extends the framework to images with unknown camera param- eters, as UniDepth (Piccinelli et al., (025) is able to estimate intrinsics directly from images. This enables training and evaluation vast 2D datasets lacking such camera intrinsics annotations.
5 EXPERIMENTS
To validate our Camera-Aware MLLM framework and its core claim that camera-awareness is essential for generalizable spatial reasoning- -e conduct three targeted evaluations. We: (i) as- sess the model's cross-camera generalization capacity specifically on ppatially-grundde benchmark its performance on established spatial reasoning benchmarks; and (iii) perform a detailed ablation study. In our experiments, we adopt VG-LLM (Zheng et al. 2025) the baseline. More implementation details are provided in Appendix
tasks; (ii)
5.1 EVALUATION ON CROSS- CASSSA SPATIALLY GROUNDED TASKS
suite of spatially-groundel tasks: single-frame 3D object detection, video 3D object detection, and video 3D visual grounding. As general-purpose models like Gemini-2.5 (Comanici et al.) 2025) fail to perform
We test our framework's generalization
3D localization, we only com-
pare against finstfinetnne baselines: Qwen2.5-VL (Bai et al. 2025) and VG-LLM (Zheng et al.] 2025). All models are trained on a large, diverse corpus inc luding ScanNet (Dai et al.
2017), ARK- itScenes (Baruch et al. 2021), Matterport3D (Chang et al. 2017). 3RScan (Wald et al. 2019)) SUN RGB-D (Song et al. 2015). Objectron (Ahmadyan et al. 2021), ScanRefer (Chen et al. (2220), and Scan2Cap (Chen et al. 202T). Models are trained on the full mixed-source data. At inference, we test ScanNet validation set, but synthetically perturb the camera intrinsics by resizing the in- put images. This directly tests model robustness against the camera geometry shifts discussed in Sec. 3.3 simulating deployment on cameras with different focal lengths.
As illustrated in Figure 6 our Camera-Aware MLLM demonstrates exceptional robustness to varia- tions in camera intrinsies. In all spatially-grounded tasks, while achieving competitive performance on the original, unscaled data, our model maintains highly consistent accuracy as camera parame- ters are altered via resizing. In stark contrast, the performance of camera- camera-agnostic baseline MLLMs degrades substantially, confirming their overfitting to the training camera's geometry. These experi- ments unequivocally demonstrate that our camera-aware approach enables true generalization across cameras with diverse parameters, critical capability that RGB-only methods fundamentally lack.
noteworthy that our framework's robustness extends to scenarios where camera intrinsics are unavailable-a common case for images sourced from the internet. This is made possible by our geometric prior distillation, where the pre-trained MMDE estimates intrinsics on the fly. As shown in Fig. (c), approach consistently and significantly outperforms namera-agnostee baselines, highlighting its superior generalization even when operating on inputs without explicit camera pa- rameters.

---

Published as a conference paper at ICLR 2026
Table 2: Comparison of MILMS' spatial reasoning performance on SPAR-Bench.
Mellino
Mg0
Metheds
SPAR-Bard ting
Theraa Lavd
67.27
15.31
12.75
14.25
15.75
35.25
76.25
52.15
65.5
33.90
72.32
92
64
6047
T6.22
93
94
71
92
301
12
50
80
16.99
45.30
13:00
13:00
37.40
34.40
23.40
24-42
24.93
30
45.11
54
ClankeA.-SSone
1542
15.24
69:00
13381
36.30
23.35
42
40.00
83
18
19.40
15.33
35.30
46:90
17:30
2010
40.00
$7.40
14440
14.90
23.05
40
35
13.16
48:44
14
74
91
56
54
24
14
24
44
SPAR-Booch (14))
25.33
20.86
16.38
16.31
13.79
05.33
27.06
51.88
61.33
78.00
63.90
59.71
51.25
01.35
96.96
01.53
20.59
19.90
11:30
31.74
30.33
25:44
18:58
1337
10.13
1141
11:31
79.34
26.13
18.08
10.35
9.11
00.14
56.17
15.05
13.25
09.63
16.33
31.78
53.33
26.14
15011
57.13
22:97
53,33
53.00
27.15
1321
8.5
1211
0.00
20.35
0.27
10.76
041
34.27
0.00
4.79
5.0
1.55
0.00
20.13
$1.76
7.74
6.25
32.14
6.37
99.52
10.43
21.52
588
GPTS
37.00
16.90
30,99
SPAR 4B
63.25
06.33
$1.59
75.25
60.90
13.00
66.47
64.00
50.00
42.34
53.90
Ozz-48
68.35
1994
8981
48.12
57.00
19.27
$5.02
63.40
17.60
35.42
60042
47.54
13000
31011
81.34
90.38
68.25
$7.50
91.70
$4.1
79.51
06.5
83.44
$1.23
Tasks split into 'low-level', "medium- level", and "high- level" tasks hased di different complexities, following SPAR
SPAR-Bench Rench authors.
Table 3: Comparison of MLLMs' spatial reasoning performance VSI-Bench.
Ohjy Count
Als. Dist
City Size
Room Size
Rel. Dia
Rel. Dir.
Route Plan
Appe. Order
Arg.
Multiple- Answer
Preprietary Generalist MLLMs:
GPT-4o
34,0
46.2
5.3
43.8
38.2
37.0
41.3
31.5
29.5
GPT-5 Chat 0807 global
29.1
52.4
36.6
68.0
55.5
49.9
47.4
33.0
49.8
Gemini-1 1.5 F.5-hash
42.1
49.8
30.8
53.5
54.4
37.7
41.0
31.5
37.8
30.9
43.6
24.6
Gemini 2.5 0617
52.7
48.2
35.6
71.3
51.7
$8.9
42.4
46.8
66.7
Open-Soureed Generalier MLLMS:
VntemV112-88
34.6
23.1
28.7
48.2
39.8
36.7
30.7
20.9
39.6
IotemV112.400
36.0
34.9
26.9
46.5
31.8
42.1
32.2
34.0
39.6
Dwen2.5V11-725
29.3
54.5
38.8
28.9
LougVILA--88
21.6
29.1
9.1
16.7
0.0
20.6
30.7
32.5
25.5
VILA-1 5-40B
31.2
22.4
24.8
48.7
22.7
40.5
25.7
31.5
32,9
LongVA-7R
12
22.8
57.4
35.5
OL.VA-OneVisinn-72
40.2
43.5
23.9
$7.6
37.5
42.5
39.9
32.5
44.6
1.MA3-7
35.8
41.9
23.5
42.2
27.1
39.4
32,0
31.4
MLLMs for Spotial Intelligonce:
SAU-LLVAA Video-7B
47.3
36.1
40.4
SPAR-SB
RynaBC-7B
45.8
58.5
25.4
54.9
42.7
44.2
38.7
30.5
VG-LLM-BB
47.3
66.0
37.8
55.2
39.2
44.6
45.6
33.5
36.4
Ours 4B
39.3
50.4
63.9
50.7
31.4
20.2
5.2 EVALUATION ON GENERAL SPATIAL REASONING BENCHMARKS
A key advantage of our framework is its re-
Table 4: Comparison of model performance on var- ious spatial understanding datasets.
markable ficxibility in handling diverse vi- sual inputs. Our model can seamlessly pro- cess both data with known camera intrin- sics, common in embodied AI, and standard RGB images or videos where such informa- absent,
CV-Beach-3D
BLINK-Spoil Syanial
Asg.
Depth
Ditt.
Avg.
Depch
Spatial Rel.
Multi View
72.7
GPT-4o
67.6
tion is absent. When intrinsics we utilize the MMDE to estimate the in- trinsics, enabling us to train a single, pow- erful generalist model for spatial reasoning on a large-scale, heterogeneous corpus. To split
Gemini 2.5 Pro
90.8
91.0
90.7
71.6
87.9
91.6
35.3
Qwes2.5. VL-BB
69.5
74.5
66.2
64.7
68.5
81.1
44.4
Qwen2.5
73.8
Quan2 V1.1222
79.1
83.0
83.2
6688
69.4
86.0
45.1
Qwen2 20om2.5-VL-7222 VL-720
$1.0
88.7
$1.0
68.6
79.8
853
40.6
SATALLaAA-Vibo-TR
78.4
48.1
this end, combine the of LLaVA-Video- 178k (Zhang et al., (2044b)
VG-ILM-8
91.3
68.4
79,8
71.3
$4.1
Ohm-48
90.7
80.8
91.5
77.0
77.4
66.4
87.2
SPAR (Zhang et al., 2025), and our rated collection peatially-groundee tasks to train generalist model on spatial reasoning.
We evaluate this generalist model range of established spatial reasoning benchmarks against state-of-the-art methods. As shown in Table 2 on BenR,Bench, which provides precise camera parameters, model achieves the highest performance, directly validating the effectiveness of our
8

---

Published as a conference paper at ICLR 2026
camera-awan design in controlled setting. As shown in Table 3 and Table 4 even general spa- tial reasoning benchmarks designed for RGB-only methods, where intrinsics are not provided, our method still achieves the state-of-the-art performance. These results demonstrate that our camera- aware approach is not narrow fix, but a foundational principle that enables robust and generalizable spatial intelligence when scaled with diverse data.
5.3 ABLATION STUDY
Table 5: Ablation study on the components our Camera-Aware MLLM framework. Performance measured on ScanNet-val x1.2 to test cross-cament generalization.
To dissect the contributions of our frame- work, we conducted ablation study on the cross-camera generalization task of single- frame 3D object detection. We evaluated performance
Components
20 Common Classes
synthetically "zoomed- in" test set set-created by rescaling and cen- trally cropping images- to simulate out- of-distribution camera with altered intrin- sics. We adopt VG-LLM (Zheng al.) 2025) as our baseline to quantify the im- provements.
Ray
Geom, Aug.
Po.25
Ro. 25
Plo.25
40.8
37.7
39.1
42.9
39.7
41.2
43.8
40.4
42.0
45.7
41.1
43.1
46.2
42.8
44.3
54.6
Our ablation in Table 5 demonstrates that neither a camera-aware architecture nor diverse train- ing data is sufficient on its own. While adopting a camera-aware architecture or employing ge- ometric augmentation alone provides moderate gains, it is their synergy that unlocks substantial improvements in generalization. This combination forces the model internalize geometric princi- ples, proving that both architectural camera awareness and data-level diversity are indispensable for achieving robust spatial intelligence MILMs.
5.4
VISUALIZATION RESULTS
Fig. qualitatively compares the 3D detection performance of our proposed Camera-Aware MILMM against camera- agnostic baselines trained with iden- tical data. The visualizations reveal a significant gap in localization accu- racy on both zoomed-in in images from ScanNet and in-the-wild images from the unseen TUM-RGBD dataset (Sturm et al. 2012). This demonstrates that by both incorporating camera parameters and being exposed to diverse camera setups via our camera-aware geomet- ric augmentation, our framework learns fundamental geometric principles. This mitigates the ambiguities that hinder baselines, leading to substantially more accurate and generalizable spatial un- derstanding and reasoning.
Zoomed ScanNer
[In-che-wiin (wid neet)
Qwer2.6-VL G-VL-3
VG-LLM-4B 48
Qurs
Figure Qualitative comparison of 3D object detection
results. Our camera-aware MLLM can robustly generalize to new camera setups.
CONCLUSION
In this work, we have demonstrated that camera-agnostic MLLMs, despite their apparent success, fundamentally flawed for spatial reasoning tasks due irresolvable geometric ambiguity. By ignoring camera intrinsics, these models fail to generalize, instead overfitting to the specific camera properties of their training data. Our proposed Camera-Aware MLLM framework mitigates this issue by injecting camera parameters, employing a geometric augmentation strategy, and distilling 3D priors. We argue for paradigm shift: build truly generalizable MLLMs that understand our 3D world, we must move beyond processing solely pixels to reasoning about the geometric principles that create them.

---

Published as a conference paper at ICLR 2026
REFERENCES
Adel Ahmadyan, Liangkai Zhang, Artsiom Ablavatski, Jianing Wei, and Matthias Grundmann. Ob- jectron: large scale dataset of object-centric videos the wild with pose annotations. CVPR, 2021.
lean-Baptisse Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, lain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. NeurIPS, 2022.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang. Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang. Humen Zhong, Yuanzhi Zhu, Mingkun Yang. Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-VL technical report. arXiv preprint ar Ar0i 13923, 2025.
Gilad Baruch, Zhuoyuan Chen, Afshin Dehghan, Tal Dimry, Yuri Feigin, Peter Fu, Thomas Gebauer, Brandon Joffe, Daniel Kurz, Arik Schwartz, and Elad Shulman. ARKitScenes diverse real- world dataset for 3D indoor scene understanding using mobile RGB-D data. In NeurIPS Datasets and Benchmarks Track, 2021.
Shariq Farooq Bhat, Ibraheem Alhashim, and Peter Wonka. AdaBins: Depth estimation using adap- tive bins. In CVPR, 2021.
Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Halber, Matthias Niebner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang. Matterport3D: Learning from RGB-D data in indoor environments. In International Conference on 3D Vision (3DV), 2017.
Boyuan Chen, Zhuo Xu, Sean Kirmani, Danny Driess, Pete Florence, Brian Ichter, Dorsa Sadigh, Leonidas Guibas, and Fei Xia. SpatialVLM: Endowing vision-language models with spatial rea- soning capabilities. In CVPR, 2024a.
Dave Zhenyu Chen, Angel Chang, and Matthias Nießner. ScanRefer: 3D object localization in RGB-D scans using natural language. In 2C2V.
Jiaxing Chen, Yuxuan Liu, Dehu Li, Xiang An, Weimo Deng, Ziyong Feng, Yongle Zhao, and Yin Xie. Plug-and-play grounding of reasoning in multimodal large language models. arXiv preprint arXiv:2403. 19322, 2024b.
Sijin Chen, Xin Zhu, Lei Yang. Xingjian Li, Jianke Yu, Jiachen Liu, and Dahua Lin. LL3DA: Visual interactive instruction tuning for omni-3D understanding. reasoning, and planning. In CVPR, 2024c.
Zhenyu Chen, Ali Gholami, Matthias Nießner, and Angel X Chang. Scan2Cap: Context-aware dense captioning in RGB-D scans. CVPR, 2021.
Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capa- bilities. arXiv preprint arXiv:2507.06261, 2025.
Angela Dai, Angel X Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. ScanNet: Richly-annotated 3D reconstructions of indoor scenes. In CVPR, 2017.
David Eigen, Christian Puhrsch, and Rob Fergus. Depth map prediction from single image using multi-scale deep network. In NeurIPS, 2014.
Chenming Guo, Tai Jin, Kun Lu, Liang Gao, Dongyan Guo, Yuliang Wang. Jun Xiao, and Yu Liu. LLaVA-3D: A simple yet effective pathway to empowering LMMs with 3D-awareness. In ICCV, 2025.
Yining Hong, Haoyu Zhen, Peihao Chen, Shuhang Zheng, Yilun Du, Zhenfang Chen, and Chuang Gan. 3D-LLM: Injecting the 3D world into large language models. In NeurlPS, 2023.
10

---

Published as a conference paper at ICLR 2026
Mu Hu, Wei Yin, Chi Zhang, Zhipeng Cai, Xiaoxiao Long, Hao Chen, Kaixuan Wang, Gang Yu, Chunhua Shen, and Shaojie Shen. Metric3D v2: A versatile monocular geometric foundation model for zero-shot metric depth and surface normal estimation. IEEE Transactions Pattern Analysis and Machine Intelligence, 2024.
Hang Hua, Qing Liu, Lingzhi Zhang, Jing Shi, Soo Ye Kim, Zhifei Zhang, Yilin Wang, Jianming Zhang, Zhe Lin, and Jiebo Luo. Finecaption: Compositional image captioning focusing wher- ever you want at any granularity. In CVPR. 2025.
Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao, Zheyu Ye, Fei Zhao, Zhe Xu, Yao Hu, and Shaohui Lin. Vision-rl: Incentivizing reasoning capability in multimodal large language models. arXiv preprint arXiv:2503.000 06749, 2025.
Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, et al. GPT-40 system card. arXiv preprint arXiv:2410.21276 2024.
Sicong Jiang. Zilin Huang, Kangan Qian, Ziang Luo, Tianze Zhu, Yang Zhong, Yihong Tang, Menglin Kong, Yunlong Wang. Siwen Jiao, et al. A survey on vision- siion-language-attonn models 2025a.
for autonomous driving. arXiv preprint 2rX.v:2506..40000
Songtao Jiang, Chenyi Zhou, Yan Zhang, Yeying Jin, and Zuozhu Liu. Fast or slow? integrating fast intuition and deliberate thinking for enhancing visual question answering. arXiv preprint arXiv:2506.. 250.::00000000066 2025b.
Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, Quan Vuong, Thomas Kollar, Ben- jamin Burchfiel, Russ Tedrake, Dorsa Sadigh, Sergey Levine, Percy Liang, and Chelsea Finn. VLA: An open-source lision-language-ee 2024.
model. arXiv preprint arXiv:2406.0922
Jiayi Kuang, Ying Shen, Jingyou Xie, Haohao Luo, Zhe Xu, Ronghao Li, Yinghui Li, Xianfeng Cheng, Xika Lin, and Yu Han. Natural language understanding and inference with milm in visual question answering: survey. ACM Computing Surveys, 57(8):1-36, 2025.
Saehyung Lee, Seunghyun Yoon, Trung Bui, Jing Shi, and Sungroh Yoon. Toward robust hyper- detailed image captioning: A multiagent approach and dual evaluation metrics for factuality and coverage. arXiv preprint arXiv:2412.15484 2024.
Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. BLIP: Bootstrapping language-iman -maua pre- training for unified vision vision-language understanding and generation. In /CML, 2022.
Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. BLIP-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In ICML, 2023.
Bin Lin, Bin Zhu, Yang Ye, Munan Ning, Peng Jin, and Li Yuan. Video-llava: Learning united visual representation by alignment before projection. arXiv preprint arXiv:2311.10122 2023.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. In NeurlPS, 2023.
Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In CVPR, 2024.
Xingyu Miao, Haoran Duan, Quanhao Qian, Jiuniu Wang, Yang Long, Ling Shao, Deli Zhao, Ran Xu, and Gongjie Zhang. Towards scalable spatial intelligence via 2D-to-3D data lifting. In ICCV, 2025.
Vaishakh Patil, Christos Sakaridis, Alexander Liniger, and Luc Van Gool. P3Depth: Monocular depth estimation with piecewise planarity prior. In CVPR, 2022.
Zhiliang Peng, Wenhui Wang, Li Dong, Yaru Hao, Shaohan Huang, Shuming Ma, and Furu Wei. Kosmos-2: Grounding multimodal large language models to the world. arXiv preprint arXiv:2306. 14824, 2023.
11

---

Published as a conference paper at ICLR 2026
Luigi Piccinelli, Christos Sakaridis, and Fisher Yu. iDisc: Internal discretization for monocular depth estimation. In CVPR, 2023.
Luigi Piccinelli, Yung-Hsu Yang, Christos Sakaridis, Mattia Segu, Siyuan Li, Luc Van Gool, and Fisher Yu. UniDepth: Universal monocular metric depth estimation. In CVPR, 2024.
Luigi Piccinelli, Christos Sakaridis, Yung-Hsu Yang, Mattia Segu, Siyuan Li, Wim Abbeloos, and Luc Van Gool. UniDepth V2: Universal monocular metric depth estimation made simpler. arXiv preprint arXiv:2502. 200:222022001000 10, 2025.
Kangan Qian, Sicong Jiang, Yang Zhong, Ziang Luo, Zilin Huang, Tianze Zhu, Kun Jiang, Meng- meng Yang, Zheng Fu, Jinyu Miao, al. AgentThink: A unified framework for tool-augmented chain-of-thought reasoning in vision-language models for autonomous driving. arXiv preprint 12xiv:2505.15298, 2025.
Shuran Song, Samuel P. Lichtenberg, and Jianxiong Xiao. SUN RGB-D: A RGB-D scene under- standing benchmark suite. In CVPR, 2015.
J. Sturm, N. Engelhard, F. Endres, W. Burgard, and D. Cremers. A benchmark for the evaluation of rgh-d slam systems. In IROS, 2012.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, L. Kaiser, and Illia Polosukhin. Attention is all you need. In NeurlPS, 2017.
Johanna Wald, Armen Avetisyan, Nassir Navab, Federico Tombari, and Matthias Niessner. RIO: 3D object instance re-localization in changing indoor environments. In ICCV, 2019.
Haifeng Wang, Bin Shi, Yiwen Zhang, Ping Huang, Songyang Wang, Hongsheng Li, and Xiaogang Wang. Chat-Scene: Bridging 3D scene and natural language with large language models. In NeurIPS, 2024.
Jianyuan Wang. Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. VGGT: Visual geometry grounded transformer. In CVPR, 2025.
Runsen Xu, Xiaolong Wang, Tai Wang, Yilun Chen, Jiangmiao Pang, and Dahua Lin. Point-LLM: Empowering large language models to understand point clouds. In ECCV, 2024.
Wei Yin, Chi Zhang, Hao Chen, Zhipeng Cai, Gang Yu, Kaixuan Wang. Xiaozhi Chen, and Chunhua Shen. Metric3D: Towards zero-shot metric 3D prediction from single image. In ICCV, 2023.
Hang Zhang, Xin Li, and Lidong Bing. Video-Ilama: An instruction-tume -tunedd audio-visual language model for video understanding. arXiv preprint Xr3v 02858, 2023.
Jiahui Zhang, Yurui Chen, Yanpeng Zhou, Yueming Xu, Ze Huang, Jilin Mci, Junhui Chen, Yu- Jie Yuan, Xinyue Cai, Guowei Huang, al. From flatland space: Teaching vision-language models to perceive and reason in 3D. arXiv preprint arXiv:2503.22976 2025.
Yichi Zhang, Ziqiao Ma, Xiaofeng Gao, Suhaila Shakiah, Qiaozi Gao, and Joyce Chai. Groundhog: Grounding large language models to holistic segmentation. In CVPR, 2024a.
Yuanhan Zhang, Jinming WU, Wei Li, Bo Li, Zejun Ma, Ziwei Liu, and Chunyuan Li. Video instruction tuning with synthetic data. arXiv preprint arXiv:2410.02711. 2024b.
Duo Zheng, Shijia Huang, Yanyang Li, and Liwei Wang. Learning from videos for 3D world: Enhancing MLLMs with 3D vision geometry priors. arXiv preprint arXiv:2505.24622. 2025.
Brianna Zitkovich, Tianhe Yu, Sichun Xu, Peng Xu, Ted Xiao, Fei Xia, Jialin Wu, Paul Wohlhart, Stefan Welker, Ayzaan Wahid, et al. RT-2: Vision-language-actionn models transfer web knowl- edge to robotic control. In CoRL, 2023.
12

---

Published as a conference paper at ICLR 2026
A DETAILED PROOF OF GEOMETRIC AMBIGUITY
We formalize the ambiguity that arises when camera intrinsics are unknown. Unless otherwise stated, we assume an ideal pinhole camera and ignore lens distortion and skew; including them does not change the conclusions.
Camera projection model. tion is
Let 3D point in the world frame be P E R The standard projec-
Jx
C2
P 10
K
R
t
K
0
Jy
cy
(3)
0
where R E (3(3) and t IR extrinsics, and + 0 is the projective scale. In the camera frame, we have
letting P. RP... + (x, Y, Z)
f2X + 7 f c.y 7
fuX Z
fuY + Cy
KP
5= 2, u
Cos V
Projected extent under a fronto-paralle configuration. Consider a vertical, fronto-parallel seg- ment of physical height H (in meters) centered on the optical axis: its top and bottom endpoints in From equation
the camera frame are up = (0, H/2, Z) and bottomm
(0, - -H/2, 2)
f (H/2)
fu( H//2
Mpp
+ Cy:
bottom
+ Cy.
(5)
Z
Z
so the image height pixels is
fy 1
Pproj
Utop
Ubottom
(6)
7
W. Coupled-scaling invariance (focal-depth & size-depth).
Analogously, for horizontal extent W we have Wproj
From equation 6 the projected height
hpmj z
fu H. It is invariant under the coupled rescaling
(su, H, Z) + (af BH, a,32)
(a, 0),
since
(aH))(HH)
M
Ippoj-
aBZ
Two canonical cases:
Focal-depth tradeoff: set Increasing focal length by À while moving the object to depth XZ leaves Apray unchanged (optical/digital zoom vs. depth).
Size-depth tradeoff: set 1. Scaling physical size and depth by the same factor leaves bpooj unchanged (small-near vs. large-far).
Thus, from single RGB image, nearby small object can be observationally indistinguishable (in projected height) from larger distant one, and a change in depth can be indistinguishable from change focal length.
Image resampling as intrinsic transformation. nates by (a .y) (without cropping) is equivalent to
Any pre-processing that rescales pixel coordi-
11) = diag(on, o, 1) (u, v, 1)
K' = diag(o., oy. 1) K.
subsequent crop with lef-left offset (Au, Av) (measured after rescaling) shifts the principal point ce OzCs Au, cy ByCy Av.
13

---

Published as a conference paper at ICLR 2026
Per-pixel rays and the role of intrinsies. dinates is, up to scale,
The back-projected ray for pixel (u, v) in camera coor-
(u cx)/ fa
u
d x K
(v fv
1
Hence the entire ray field {d(u, v)} depends on (fz: fyr Cz Cy), If intrinsics unknown, two images that appear similar can correspond to different ray bundles, encouraging camera-specific shortcuts rather than geometry that transfers across cameras.
Scope and assumptions. The invariance in equation 6 is derived for a fronto fronto-parallel segment and concerns the observable projected extent (e.g.. Rgro)). For non-fronto-pat tronto-parallel shapes or general scenes, pixel-wise equality need not hold under the above transformations; nevertheless, the metric identifiability issue at the level of absolute size and depth persists: with monoeular RGB and un- known intrinsics, the mapping in equation 4 is homogeneous in (X, Y. Z), metric depth and size cannot be uniquely recovered without additional metric priors, multiple views, or known intrinsics. Providing K (or its sufficient statistics, such as per-pixel ray directions) is therefore necessary information channel to resolve the depth ambiguity and enable cross-camers generalization.
B IMPLEMENTATION DETAILS
Our work builds upon the VG-LLM-4B (Zeng al. 2025) framework, which we adopt our base- line. The VG-LLM-4B model integrates Qwen2.5-VL.33 VL-3B (Bai et al. (025) with VGGT-1B (Wang) al., 2025) for 3D geometry feature extraction. To further enhance the model's geometric un- derstanding ability, we employ UniDepth v2 (Piccinelli et al. 2025) to distill and inject rich 3D geometric priors into the framework. We train the model for one epoch using the Adam optimizer with a warmup ratio of 0.03. The learning rate is gradually increased to le-5 and subsequently de- cayed. The batch size is set 64 in total. During training, the visual encoder Qwen2.5-V 3D geometry encoder (VGGT), and UniDepth v2 are frozen, while the MILM,, camera ray embed- ding. and the 3D geometric prior distillator remain trainable. All experiments are conducted on H100 80G GPUs.
We adopt a mixed dataset for training. For spatially-grounded tasks (Sec 5.1 our model trained ScanNet (Dai et al. 2017). ARKitScenes (Baruch et al.) 2021
Matterport3D (Chang et al.
2017). 3RScan /Wald et al.| 2019), SUN RGB-D (Song et al. 2015). ). Objectron (Ahmadyan et al,
2021 ScanRefer (Chen al. (022)). and Scan2Cap (Chen al. (02TT. For spatial reasoning tasks
(Sec 5.2h. our model is trained on the I.L.VA-Houm split of -LVV-Viido- 17dde (Zhang et al.]
2024b), SPAR (Zhang al., 2025), well as the aforementioned curated collection of spatially- grounded datasets. For ablation studies 5.3 our model is trained on the task of single-frame 3D object detection based the ScanNet (Dai et 2017).
Table 6: The prompt for 3D video object detection.
Xuser: <image>< -eee--iiinnne--iii image>< image>...
Detect the 3D locations for the following categories: curtain, bathtub, towel, doorframe, and bar. Output json list where each entry contains the object name in "label" and its 3D bounding "box 33",, The 3D bounding box format should be [x_center, y yeenter, zeneter, x x_size, y sisize, z z_size, yaw, pitch, rolll].
Xaoane:
json
("label": "curtain", "bbox_ 3d": [-0.5, -0.0, 0.7, 0.4, 2.0, 225, -291],
("label": "bathtub", "bbox_3d": [-0.3, -0.6, -1.2, 2.6, 0.8, 0.7, 200 1.0 2.71).
To prevent hallucinations in 3D video object detection (such the inclusion of categories that absent from the scene), we explicitly provide the categories the input. An example prompt for 3D video object detection is presented in Table
14

---

Published as a conference paper at ICLR 2026
C LLM USAGE DISCLOSURE
Large Language Models (LLMs) were utilized as a general-purpose assist tool during the preparation of this manuscript. Specifically, LLMs aided in polishing the language and improving the overall readability of the paper, assisting with various writing tasks, and also providing support for coding- related aspects of the research.
The authors confirm that any LLM-generated content was based on the authors' original ideas and experiment results, thoroughly reviewed and verified by the authors, and accurately reflects the intended meaning. LLMs were not used to autonomously complete any part of the paper.
15


Reviews:
[{"review_id": "kUUghlJaYJ", "reviewer": "Reviewer_spFi", "summary": "This paper introduces a camera-aware MLLM framework that integrates camera embedding, geometric prior distillation, and camera-aware augmentation into the existing MLLM architecture.  \nExtensive experiments are conducted on spatial reasoning benchmarks for MLLM models, and ablation studies for each proposed component are provided.", "questions": "- The formatting of Tables 1 and 3 could be improved. It would also be beneficial to include citations for the methods compared in the tables.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- The writing is clear, and the logical flow is easy to follow.  \n- The benchmark coverage is extensive.  \n- Incorporating ray embedding, camera-aware augmentation, and distillation from 3D models is a reasonable and well-motivated approach.", "weaknesses": "- Figure 1 may be misleading. The caption states, “There is no way to know unless I know the camera intrinsics!” However, even with camera intrinsics, it is still not analytically possible to determine the exact 3D location from a single RGB image. It would be helpful to clarify that priors are still needed to estimate the 3D position.  \n- The performance reported in Tables 2 and 3 is comparable to that of other spatial MLLMs, which makes the paper’s claims less convincing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Qw1NVhC4xf", "reviewer": "Reviewer_L6n4", "summary": "The paper argues that for spatial intelligence MLLMs need camera intrinsics as input to the model — prior spatial MLLMs do not use this input and thus are not robust to changes in camera parameters. The paper first quantifies this problem by showing lack of generalization on 3D object detection on ScanNet when models do not use camera information as input. Next, it proposes to inject this information by encoding the cameras rays with the visual features; doing augmentations; and training on wide varieties of datasets. In experiments, it shows that their method with camera information works better than existing methods, and the ablations show that all proposed changes help", "questions": "N/A -- no major questions or concerns", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper is well-written and easy to follow\n- The idea in general makes sense and the results look good\n- The experiment section is well designed and answers most research questions raised by the paper", "weaknesses": "No major weaknesses per se, just some comments regarding the writing and presentation of the work\n\n1. About Table 1: This table nicely shows that prior VLMs do not benefit from multiple dataset training and are susceptible to zooming-in/out operations. I was expecting the paper to show at the end that this problem is now resolved with the proposed technique and the paper does show it but it wasn’t straightforward to make this connection. Specifically, Table-1, Figure-6 and Table-5 are connected by this evaluation — yet Table-1 uses top 31 classes while figure-6 and table-5 uses 20 classes. Additionally, an explicit mention of this connection between Table-1 and Figure-6 and 5 will help. \n2. (Minor comment): The section 3.3 describes the scale-depth ambiguity in a lot of words which is an extremely well-known computer vision phenomenon. The authors can consider making this section short and crisp\n3. Table-2 is missing highlights on best working method numbers, making it harder to parse quickly. Additionally, some additional descriptions of the datasets and benchmarks will help. For eg. It is unclear what low, medium and high in Table-2 mean.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1t3DUsiJzS", "reviewer": "Reviewer_VBue", "summary": "This paper identifies a fundamental limitation in RGB-only Multimodal Large Language Models (MLLMs) for spatial reasoning tasks: by ignoring camera intrinsic parameters, these models cannot disambiguate geometric properties from camera perspectives, leading to poor cross-camera generalization. The authors propose a Camera-Aware MLLM framework that addresses this issue through three key novelties: (i) dense camera ray embeddings that condition each visual token on its corresponding camera geometry, (ii) camera-aware data augmentation that synthetically varies camera parameters during training, and (iii) geometric prior distillation from a pre-trained monocular depth estimation model (UniDepth v2).", "questions": "See weaknesses.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The paper demonstrates that camera-agnostic MLLMs fail to generalize across different camera parameters, highlighting a critical but overlooked limitation in current spatial reasoning approaches.\n2. The proposed framework combines three complementary components to resolve the identified ambiguity, with experimental results showing improvements in cross-camera generalization on multiple spatially-grounded tasks.", "weaknesses": "1. Table 5 does not include \"Prior Distillation only\" or \"Geom Aug + Prior Dist (w/o Ray Emb)\", making it hard to isolate the contribution of camera ray embeddings. Given that UniDepth v2 is pretrained on 10M+ RGB-depth pairs, the geometric prior distillation likely contributes the majority of improvements, but this is never quantified.\n\n2. Outdated and incomplete baselines:\n* Table 2/3/4 use Gemini-1.5-Flash/Pro (Feb 2024) instead of Gemini-2.5 or later versions\n* Table 4 is missing Qwen2.5-VL-7B and Qwen2.5-VL-72B, which are natural baselines given the authors' use of Qwen2.5-VL-3B\n* No comparison with Claude 4.5 Sonnet, GPT-5 or other recent strong MLLMs\n\n3. Table 1 and Figure 6 only test uniform image rescaling (0.8×, 1.2×), which is an extremely limited simulation of camera variation. Real cross-camera generalization should involve different camera models with varying focal lengths (wide-angle vs telephoto) and different aspect ratios and principal point locations.\n\n4. Tables 3-4 show that the proposed camera-aware design provides little to no benefit on standard spatial reasoning benchmarks. The results suggest camera-awareness is only beneficial for a narrow subset of spatially-grounded tasks, not a fundamental requirement for spatial reasoning.\n\n5. Missing computational cost analysis (inference time, memory overhead of running UniDepth v2)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "X81qd8AU9x", "reviewer": "Reviewer_z9kB", "summary": "This paper investigates the limitations of \"RGB-only\" multimodal large language models in spatial reasoning tasks, particularly their lack of generalization across camera viewpoints. The authors attribute this to the absence of camera intrinsics, which they argue introduces geometric ambiguity and leads to overfitting. To address this, the paper introduces a Camera-Aware MLLM framework that incorporates dense camera embeddings, camera-aware data augmentation, and geometric prior distillation from a 3D vision model.", "questions": "See Weaknesses", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "(1) This paper is well-motivated. It identifies a fundamental limitation in existing MLLMs for spatial tasks, based on both theory and empirical analysis. What’s more, the cross-camera generalization experiments are particularly compelling and clearly demonstrate the value of the proposed method.\n\n(2) The proposed Camera-Aware MLLM framework is evaluated through a series of experiments. The results show consistent improvements over camera-agnostic baselines, suggesting that incorporating camera parameters can enhance the robustness of spatial MLLMs.", "weaknesses": "(1) Limited discussion on real-world applicability. The paper would benefit from a discussion about the real-world scenarios where camera intrinsics may be unknown, unreliable, or hard to estimate depth. \n\n(2) The ablation study in Table 5 lacks the “Only Prior Distillation” setting, which I consider important for understanding the individual contribution of geometric prior supervision.\n\n(3) It remains unclear whether the improvements stem from the proposed model itself or simply from the availability of additional 3D geometric information. It could be clarified by comparing with methods that directly utilize depth maps or incorporate depth-aware features into the model.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Oral)