PLANA3R: Zero-shot Metric Planar 3D Reconstruction via Feed-forward Planar Splatting
Authors: Changkun Liu
Bin Tan
Zeran Ke
Shangzhan Zhang
Jiachen Liu
Ming Qian
Nan Xue
Yujun Shen
Tristan Braud
Venue: NeurIPS 2025 poster ()
Keywords: 3D Reconstruction
Planar Reconstruction
Plane Segmentation
Primary area: applications
TL;DR: We present PLANA3R, a zero-shot method for metric Planar 3D Reconstruction from unposed two-view images.

Abstract: This paper addresses metric 3D reconstruction of indoor scenes by exploiting their inherent geometric regularities with compact representations. Using planar 3D primitives -- a well-suited representation for man-made environments -- we introduce PLANA3R, a pose-free framework for metric $\underline{Plana}$r $\underline{3}$D $\underline{R}$econstruction from unposed two-view images. Our approach employs Vision Transformers to extract a set of sparse planar primitives, estimate relative camera poses, and supervise geometry learning via planar splatting, where gradients are propagated through high-resolution rendered depth and normal maps of primitives. Unlike prior feedforward methods that require 3D plane annotations during training, PLANA3R learns planar 3D structures without explicit plane supervision, enabling scalable training on large-scale stereo datasets using only depth and normal annotations. We validate PLANA3R on multiple indoor-scene datasets with metric supervision and demonstrate strong generalization to out-of-domain indoor environments across diverse tasks under metric evaluation protocols, including 3D surface reconstruction, depth estimation, and relative pose estimation. Furthermore, by formulating with planar 3D representation, our method emerges with the ability for accurate plane segmentation. The project page is available at: \url{https://lck666666.github.io/plana3r/}.

PLANA3R: Zero-shot Metric Planar 3D Reconstruction via Feed-Forward Planar Splatting
Changkun Liu Liu
Bin Tan 2-
Zeran Ke Shangzhan Zhang2.4 Jiachen Liu
Ming Qian
Nan Xue Yujun Shen? Tristan Braud'
'hee Hong Kong University of Science and Technology Ant Group
Wuhhn University
*Zhejiang University
Thhe Pennsylvania State University
Two-view Planar Reconstruction
Single e-view Planar Reconstruction
Plane Rec.
Texture Planar Primitives
3D Plane Instances
2D Plane Seg.
Figure 1: Our proposed PLANA3R learns to predict planar 3D primitives and metric-scale relative poses, providing compact 3D representation of two-view input images with accurate pose estimation, surface geometry, and semantically meaningful 2D & 3D planar segmentation-al in one pass. As side output, our model also performs well on single-view planar 3D reconstruction.
Abstract
This paper addresses metric 3D reconstruction of indoor scenes by exploiting their inherent geometric regularities with compact representations. Using planar 3D primitives a well-suited representation for man-made environments we introduce PL.ANA3R, a pose-free framework for metric Planar 3D Reconstruction from unposed two-view images. Our approach employs Vision Transformers extract set of sparse planar primitives, estimate relative camera poses, and supervise geometry learning via planar splatting, where gradients are propagated through high-resolution rendered depth and normal maps of primitives. Unlike prior feedforward methods that require 3D plane annotations during training, PLANA3R learns planar 3D structures without explicit plane supervision, enabling scalable training on large-scua scale stereo datasets using only depth and normal annotations. We validate PLANA3R on multiple indoor-scene datasets with metric supervision and demonstrate strong generalization to out-of-domain indoor environments across diverse tasks under metric evaluation protocols, including 3D surface reconstruction, depth estimation, and relative pose estimation. Furthermore, by formulating with planar 3D representation, our method emerges with the ability for accurate plane segmentation. The project page is available at: https: //1ck666666. github. i plana3r/.
'Equal Contribution. Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Introduction
Indoor environments are the primary setting where humans spend most of their daily lives. Yet, computationally creating digital twins of these 3D spaces from captured images remains challenging. Factors such as the difficulty of accurate camera pose estimation from indoor images [28, 11, 1] and structural distortions in the resulting 3D reconstructions [22, 12, 21] hinder the development of robust, accurate, and user-friendly solutions for replicating indoor scenes in the digital world.
As indoor scenes are typically rich in planar structures such floors, ceilings, and walls, well as planar furniture like tables and cabinets, planar primitives are well-suited representations for the accurate 3D reconstruction of indoor scenes. As result, there has been significant interest among the research community in planar 3D reconstruction in recent years. Planar reconstruction approaches include feedforward solutions in monocular [40, 16, 27, 24, 18, 42] and two-view [11, I, 28] settings, and per-scene optimization approaches [29, 38, 3, 9] that leverage posed multi-view inputs with the assistance of the feedforward models were studied. However, these approaches face two key
limitations:
Annotation dependence for feedforward methods: Learning feedforward models [36, 24, 28] typically requires accurate plane masks and 3D plane annotations from monocular or binocular inputs. This reliance on dense annotations limits generalizability and thus impairs zero-shot performance
iitoo--distribution data.
Pose dependence for optimization-based methods: Per-scene optimization techniques [29, 9, 38, 3] depend on accurately posed multi-view, always available and would lead to either undermining reconstruction quality or limited usage scenarios. Furthermore, these methods cannot handle several sparse pose-free views or just a pair pose-free views.
images, which are not
This paper focuses on two-view planar 3D reconstruction, one of the most fundamental tasks in structured indoor scene modeling. We aim to eliminate the reliance dense plane-level annotations and accurate multi-view camera poses, addressing the limitations discussed above. Inspired by recent advancements in 3D foundation models [33, 14, 31, 32, 30, 37, 41], we show that feedforward, pose-free, and zero-shot generalizable planar 3D reconstruction from unposed stereo pairs both feasible and effective through our proposed method, PLANA3R.
PLANA3R is built upon Vision Transformers [7], which process stereo image pairs jointly learn 3D planar primitives and relative camera poses at scale. Rather than relying on explicit planar 3D annotations, we leverage depth and normal maps?, which more readily available in existing two-view datasets at scale, as supervision signals to train the Transformer-based model. To this end, we adopt the differentiable planar rendering technique from PlanarSplatting [29] to generate high-resolution rendered depth and normal maps, and then compare them with the corresponding ground truth guide learning through gradient-based optimization. Once the model trained, our method generates a set of 3D planar primitives that approximate indoor scenes far more efficiently than per-scene optimization methods [29, 9, 38, 3]. As shown in Fig. our trained PLANA3R computes metric pose and planar primitives in a single feed-forward inference, resulting in compact representation of indoor scenes with accurate geometry and meaningful semantics.
Given that indoor environments are constructed by humans, and the dimensions of human-scale objects tend to follow similar distributions across different scenes, consider that planar 3D representation inherently possesses the potential to predict metric 3D geometry directly. Therefore, during training data preparation, we pay particular attention to preserving the metric scale of both depth maps and relative camera poses. In total, four million training data from ScanNetV2 [4], ScanNet++ [39], ARKitScenes [5], and Habitat [23] have been prepared to train our PLANA3R model with metric scales.
PLANA3R exhibits strong generalization ability to out-of-domain indoor environments in terms of depth estimation, 3D surface reconstruction, and relative pose estimation. Benefiting from the planar-based representation used, our model also empowers the capacity to provide promising instance-wise planar segmentation, enriching the semantics of 3D) reconstruction without the need for
FFor the dataset that only contains depth maps, the Metric3Dy2 [10] is applied to generate pseudo labels as the normal maps. Please refer to Sec. 4.2 for details.
2

---

plane masks. Furthermore, we argue that indoor scenes are particularly well-suited for training metric 3D vision foundation models. The regular geometry and semantic consistency indoor environments provide an ideal context for developing models that generalize across scenes and accurately estimate metric information.
2 Related Work
In this section, we briefty summarize the recent developments on the 3D reconstruction of indoor scenes in two aspects of planar representation and feedforward modeling.
2.1 Indoor Planar 3D Reconstruction
Given the strong planarity of indoor scenes, planar surface reconstruction has been extensively studied in the literature [17, 16, 24, 1, 40, 11, 28, 34, 36, 29, 9, 38]. Among existing methods, feedforward approaches 27, 36, 28, 11] are particularly appealing due to their simplified computation pipelines powered by neural networks. However, most of these methods focus in in-oomain testing using small or moderate-scale datasets. major limitation is the scarcity of large-scale, high-quality 3D planar annotations, which constrains supervision and limits the generalization ability of models trained such data.
For instance, methods like SparsePlanes [11]. PlaneFormers [l], and NOPE-SAC [28] rely on single-view 3D plane annotations and two-view 3D plane correspondences as supervision, restricting usable data sources to ScanNetV2 [4] and Matterport3D [2] datasets and requiring complex data preprocessing pipelines. To address this, our work improves data accessibility by leveraging PlanarSplatting [29] an alternative the manually-amnottee planar supervisions. This makes it possible to train models on large-scale 3D datasets by learning sparse planar primitives without requiring explicit plane-level annotations.
2.2 Feed-forward Stereo Foundation Models for 3D Reconstruction
The rise of large-scale training has significantly transformed the landscape of 3D reconstruction [33, 14, 32, 41, 31, 30, 37J. Recently, DUSI3R [33] introduced feedforward framework that predicts dense point clouds from stereo image pairs without requiring known camera poses. Building on this, MAS(3R [14] further enhances performance by generating point clouds in metric space. Several follow-up works have since explored ways improve both scene geometry estimation [32, 30] and camera pose prediction [6, 41]. However, these methods typically assume that scene geometry can be effectively represented by densely sampled 3D points, which introduces redundancy when modeling structured environments. In contrast, we adopt more abstract planar primitives to express structured scenes with the best pursuit of compactness and efficiency representation.
While our method shares architectural similarities with DUSt3R [33] and MAS13 [14], notably the use of Vision Transformers [7], we depart from their dense point-based representations. Instead, we focus on exploiting the strong planar regularities characteristic of indoor scenes by adopting a structured planar representation. We demonstrate that this formulation, based on the use of sparse planar primitives, can represent structured scenes more compactly and efficiently while also enabling the extraction of instance-level planar semantics.
3 Method
PLANA3R is a transformer-based model for two-view metric 3D reconstruction, using sparse 3D planar primitives a scene representation. Given two pose-free images from the same scene, along with known intrinsics, PLANA3R predicts their relative camera pose and infers a set of 3D planar primitives in single feed-forward pass. We build upon planar primitives introduced in PlanarSplatting [29], and leverage its CUDA-based differentiable renderer for supervision. PlanarSplatting is core component of PLANA3R, providing ultra-fast and accurate reconstruction of planar surfaces in indoor scenes from multi-view images. Instead of detecting or matching planes in 2D 3D, it directly splats 3D planar primitives into dense depth and normal maps via differentiable, CUDA-acceleraal accelerated rasterization.
3

---

This allows PLANA3R to be trained directly from monocular depth and normal labels, without requiring explicit plane annotations. Sparse planar primitives offer more compact and semantically meaningful alternative to dense point clouds or 3D Gaussian Splatring (3DGS) [13], particularly in structured indoor environments. They approximate scene geometry with high fidelity and support the rendering of dense, consistent depth and normal maps. Moreover, the explicit structure of primitives facilitates downstream tasks such as plane extraction and segmentation.
The predicted primitives can be further merged into coherent planar surfaces to support both 3D reconstruction and segmentation. We detail the task setup and notation in Sec. 3.1, describe our architecture in Sec. 3.2, outline training objectives in Sec. 3.3, and explain the primitive merging process in Sec. 3.4.
3.1 Task Setting and Notation
An overview of PLANA3R is shown in Fig. 2. The input consists of two images 12, 1 E RX with camera intrinsics K and K². Our goal is to train a network F outputs a set of sparse 3D planar primitives and the 6-DoF relative camera pose Prat. Pas is represented by the quaternion q R+ and translation E R
Trnneorrmr
Priméive Selection
Desader1
Inbersaion
Encoder
Sharlng
Transoemer
Impur
Planar Reconstracionn
A set of planur primitives
Figure 2: Overview of our PLANA3R. Given two images captured from the same scene, PLANA3R outputs of 3D planar primitives and 6-DoF relative camera pose Piet in metric scale. PLANA3R does not perform per- ver-pixel primitive prediction. Instead, it employs deconvolution network to predict primitives at two distinet resolutions, based on the patch divisions from the ViT encoder. During both training and inference phases, we selectively integrate low-resolution and high-resolution primitives, resulting in a highly compact and expressive representation. Note that even for high- resolution primitives, their total number is only !! x W
g which remains highly sparse.
F: (('( 1122 (( r ) j=1² Pret.
(I)
where q denotes the quaternion of a planar primitive generated from image 7', represented in the coordinate frame of the camera j. d' E R4 denotes the depth of planar primitive center associated with image I', N, denotes the number of planar primitives associated with image I'. plane radii. r' refers to the radii array of planar primitive associated with image I'. In our training setup, we fix = 1, treating I as the reference frame. Accordingly, the ground-trutt truth camera pose of I is set the 4 x identity matrix. Then, the primitive centers c² E g from both I and 1 are transformed into the coordinate frame of the camera j = 1 using the corresponding intrinsics K depth d', and camera poses. Following PlanarSplatting [29], we define the same positive direction of the X-axis/Y-axis of the 3D planar primitive and calculate the normal of the planar primitive n, R as:
r. = (rzz r2), r, f E R denotes
n R H(,,)00,0,1
where R(qx) converts q to the rotation matrix.
Input images {1') )= are first encoded in a Siamese fashion using a ViT encoder [7], producing feature maps {F' )- R W. Pas. These features are then processed by two transformer decoders with cross-attention to produce low-resolution decoder embeddings |Gi,
R f/ H. xDx, separate head regresses the relative camera pose Pra from the concatenated low- resolution features (G ) This partial architecture of our network J is inspired by some stereo 3D vision models [33, 14, 6, 35]. The core innovation of our method lies in the sparse primitive prediction architecture outlined in Sec. 3.2 and 3.3.

---

Compared to per-pixel 3D Gaussian Splatting and dense 3D point regression methods, our approach leverages the inherent compactness planar primitives. Many pixels naturally form coherent planar regions in structured indoor environments, making per-pixel representations unnecessarily redundant. Although non-planar or texture-rich areas may require finer approximations with smaller primitives, such cases are relatively rare. To achieve a more compact and efficient geometric representation using fewer primitives, we propose a hierarchical primitive prediction architecture (HPPA) to fit the scene using planar primitives, enabling compact modeling of scene geometry with sparse primitives.
3.2 Hierarchical Primitive Prediction Architecture
From {Gimm |i,,2, three regression heads first predict low-resolution patched planar primitives {(d,, r
H W )= To obtain higher-resolution features, we apply deconvolution network to upsample 1 generating (Gighh= E W Das, The same three heads then
# #
regress high-resolution patched planar primitives {dd r q ) 11 j=1' The primary challenge then becomes determining which image regions require low-resolution planar primitives and which benefit from higher-resolution primitives. To address this, we propose simple yet effective heuristic that avoids the need for additional learning-
Our approach is based on the observation that regions exhibiting significant variations in normals require a greater number of smaller planar primitives for accurate fitting, whereas areas with minimal changes in normals can be effectively represented by fewer planar primitives with larger radii. For each input image, using d in predicted primitives, we derive the patched depth maps: D patch
low E TR # x #
and D parch high IR. W Similarly, using 9r predicted primitives and Eq. (2), two different patched normal maps are obtained: Npath Rax # TO and N high R3 $ $. We compute the gradient
magnitude for each pixel in the low-resolution predicted normal patches Noowh of size H w 16 and
selectively use the high-resolution planar primitives of size x W. only for those pixels whose gradients exceed a predefined threshold 9th during rendering, To combine the low- and high-resolution primitives, we use binary masks to merge only the valid patches from both resolutions, rather than directly using all predicted primitives. We provide more details in Fig. 12 (b).
5
3.3 Training Losses and Training Strategies
For input images (=,,, PLANA3R generates planar primitives at both low and high resolutions. Specifically, the low-resolution primitives represented as { {dd y 1x q.d))
18 x #
1i=12; while the
high-resolution primitives given by {(d", r {) 1=,2 j=1' Due to the randomness of initialization at the beginning of training. the initial planar primitives often lie outside the viewable range defined by the ground-truth (GT) camera poses. As result, directly applying the differentiable
# 4
planar primitive rendering from PlanarSplatting [29] becomes ineffective for optimizing these learnable primitives. To address these challenges and facilitate training, we introduce patch loss designed to stabilize primitive positioning and orientation:
Caach 01 (Nrutch)
N st
+ tn ||uutch N s1 + 02 Dracch - DD
(3)
where (low, high}, on and 02 are loss weights. Here, D and N are derived by resizing the ground-truth depth maps De E B and normal maps Ne E R dimensions x
W. and 5 x g respectively. The patch loss the initial warm-up training stage aids in stabilizing the spatial positioning and orientation of the primitives.
After the warm-up phase, we introduce rendering loss. We render depth and normal maps from both primitive sets. Unlike the patch loss in Eq. (3), which supervises only depth and orientation, the rendering loss in Eq. (4) compares rendered and ground-truth maps using PlanarSplatting rasterization at full resolution (H x W). This enables gradient flow to refine primitive radii and improves geometric fidelity. Specifically, Neener E R3x xW and Deader & IR. R xW are rendered
from H. W. low-resolution primitives, while Nrealer E H and remder E Hx originate high
I6 Tg
X
high
from x W
g high-resolution primitives. As discussed in Sec. 3.2, during the inference stage,
5

---

rendering all regions of the image using all high-resolution primitives is redundant. Therefore, during training, we select some low low-resolution primitives and some high-resolution primitives for rendering. We compute the gradient magnitude for each pixel in low-resolution Maalch and use high high-resolution planar primitives only for those pixels whose gradients exceed predefined threshold Iū during rendering. Nrenddr
low
selected E x23xHx and D reedder selected E IxxxxW are rendered from selected primitives,
T6 N 8
To optimize these learnable planar primitives, our PLANA3R is trained in a supervised manner based on the differentiable planar primitive rendering in PlanarSplatting:
Crender = B1 || (Ntender)
No
+ 3 N N
" + 32 Drnner D
I
(4)
where * flow, high, selected} and 81, .2 balance the loss magnitudes for stable training.
For the predicted relative pose Pra [t. q.. we use MSE loss and relative angle loss to provide supervision:
_ 71 ||t - -t/1 +
qot
q
+
+
tat
(5)
[|lll
||t]
t
where 71 72: 73 are loss weights for different items.
Note that our model focuses on metric reconstruction. Therefore, we do not apply normalization to the training labels. Instead, we use metric depth maps and metric poses for supervision in Eq. (3), Eq. (4), and Eq. (5).
3.4
3D Plane Merge
Given a pair input images, once the collection of 3D planar primitives is predicted, we perform similar merging in PlanarSplatting by setting thresholds for the normal and distance errors between each pair of primitives. This process enables the extraction of semantic information for each plane and yields the final planar surface reconstruction. We provide more details in Appendix A.2.
4 Experiment
4.1 Implementation Details
We initialize the VIT encoder and the transformer decoder's part of PLANA3R model with DUSt3R's pre-trained 212-DPT weights. Training performed using the Adam optimizer [19] with a learning rate starting 1 10-4 and decaying on NVIDIA H20 GPUs, with a per-GPU batch size of 6. Training starts with a one-epoch warm-up phase that optimizes only the losses in Eq. (3) and Eq. (5), followed by 10 epochs incorporating all three losses at an input resolution of 512 x 384. During both training and testing, we set the gradient threshold It for merging high- and low-resolution primitives 0.5. Additional details are provided in the supplementary materials.
10-6 The model is trained for total of 256 GPU-days
4.2 Datasets
Since PLANA3R targets structured indoor scenes, we train on a combination of four public indoor- scene datasets: ScanNetV2 [4], ScanNet++ [39], ARKitScenes [5], and Habitat [23]. From these datasets, we construct approximately four million image pairs. Pseudo GT normal maps are generated using Metric3Dv2 [10], while GT depth maps are directly taken from the datasets. For evaluation, we use SeanNetV2, Matterport3D [2], NYUv2 [20], and Replica [26] as test sets. Among these test sets, except for Sea the remaining three datasets demonstrate the generalization capability of our model across different datasets. We also evaluate on 7Scenes [25] in Appendix A.
4.3 Baselines and Evaluation Metrics
We evaluate our PLANA3R against state-of-the-art (SOTA) planar reconstruction methods across multiple tasks, including 3D reconstruction, pose estimation, depth estimation, and plane segmentation, using diverse scene types and image pairs. These comprehensive evaluations

---

demonstrate our method's superior performance in both geometric accuracy (metric 3D reconstruction, depth estimation, and two-view relative pose estimation) and semantic understanding (plane segmentation). We primarily compare PLAANA3R with two-View and single-image planar 3D reconstruction methods [11, 24, 28]. We also compare with the currently popular stereo metric vision foundation model, MASt3R [14], which uses dense point clouds as the scene representation.
4.3.1 Metric Two-view Reconstruction
We evaluate the geometric quality of reconstructed 3D planes using Chamfer Distance and F-score on ScanNetV2 and Matterport3D datasets. Given pair of input images, PLANA3R predicts collection of planar primitives, which are subsequently merged into the planar surface reconstruction using the approach described in Sec. 3.4.
We conduct extensive experiments across a wide range of scenes to evaluate the effectiveness of our method in two-view 3D reconstruction and relative pose estimation. For ScanNetV2, follow the training and testing splits defined by NOPE-SAC [28], evaluating 4051 image pairs from 303 scenes. For Matterport3D, we evaluate 6083 image pairs from 13 challenging scenes. As shown in Tab. PLANA3R achieves SOTA performance on ScanNetV2. Remarkably, despite never being trained on Matterport3D, PLANA3R outperforms prior planar reconstruction methods [11, 24, 28] that were specifically trained on this dataset, highlighting the strong zero-shot generalization capability of PLANA3R across diverse indoor environments.
We further evaluate relative camera pose estimation on ScanNerV2 and Matterport3D using the same image pairs evaluated in reconstruction. Pose accuracy is measured by the metric translation error (in meters) and rotation eπror (in degrees). As shown in Tab. I, both MASt3R and our PLANA3R significantly outperform prior learning-bassee based planar reconstruction methods [28, 11, 1] terms of pose estimation accuracy.
Input Image
Input Image
NOPE-SAC
Ground Truth
Figure 3: Comparisons of two-view 3D planar reconstruction the ScanNetV2 [4] (the first row) and the Matterport31) [2] (the last two rows) datasets.
4.3.2 Metric Monocular Depth Estimation
Directly rendering depth from the predicted planar primitives provides an effective means of evaluating the quality of our geometric fitting. This process does not require merging the primitives and can be performed with a single feed-forward pass. We evaluate the metric monocular depth estimation using the NYUv2 [20] dataset. For our experiments, we use the image splits defined in [24, 15] with 654 test frames. We input two identical images into PLANA3R and use the CUDA- based differentiable planar primitive rendering module in PlanarSplatting to generate a depth map of the predieted primitives from one of the views.
Since our PLANA3R has never seen the scenes of NYUv2 during training, this dataset can well demonstrate the generalization ability of our model for out-of-domaii images. As shown in Tab. 2, PLANA33 demonstrates strong zero-shot metric depth estimation performance, surpassing both prior learning-based planar reconstruction methods and the metric 3D vision foundation model MAS(3

---

Table 1: Quantitative comparison of two-view planar reconstruction and relative camera pose estimation. The best results are bold.
Translation (m)
Rotation (
Mothod
Chamfer
F-score?
Med.
Mean
<=1
<=0.5
<=0.1
Med.
Mean
<=30
<=15
<=5
ScanNetV2 dataset
SparsePlunes [1]]
0.56
0.81
73.7
44.6
15.46
33.38
70.5
48.7
PlaneFormers [1]
0.55
0.81
75.3
45.5
14.34
32.08
73.2
52.1
NOPE-SAC
0.41
0.65
59.1
8.27
22.12
25.03
61.86
MASOR [14]
97.65
47.37
2.17
6.67
84.37
74.92
OUTS
0,07
0.13
98.62
97.16
67.91
2.01
3.16
99.23
98.89
93.14
0.11
92.52
Matterport3D dataset
SpansePlanes [11]
0.62
1.10
67.32
40.67
3.67
7.27
22.
84.15
73.60
36.94
0.47
48.59
NOPE-SAC (28)
0.53
0.91
73.60
47.82
2.79
13.81
89.71
87.41
69.83
0.38
54.96
MAS03 [14]
0.41
0.59
85,20
58.87
8.55
0.98
4.66
96.58
96.09
92.59
0.49
30.01
ours
0.24
0.45
92.78
78.83
15.07
2.00
4.49
98.01
97.19
89.82
0.32
56.63
These results highlight the potential of using sparse 3D planar primitives as an efficient and effective scene representation, in contrast to methods relying on heavy DPT heads [21] and dense point clouds.
Table 2: Quantitative comparison of metric monocular depth estimation the NYUv2 dataset. The best results are bold.
Method
PlaneNet [15]
PlaneAE
PlaneRCNN
PlaneTR [27]
PlaneRecTR [24]
MAS [14]
Ours
Rel
0.239
0.205
01183
0.195
0.157
0.152
0.132
logio
0.124
0,097
0,076
0.095
0.073
0.058
0.059
RMSR
0.913
0.820
0.619
0.803
0.547
0.51
0.463
53.0
61.3
71.8
63.3
74.2
83.0
86.4
by
78.3
87.2
93.1
88.2
94.2
95.6
96.3
8g f
90.4
95.8
98.3
96.1
99.0
98.7
98.4
4.3.3 3D Plane Segmentation
The previous experiments demonstrate the geometric accuracy of model. Here, we show that PLANA3R can perform zero-shot plane-level semantic segmentation without plane annotations. By merging predicted planar primitives, infers semantically meaningful 3D planes. We evaluate segmentation quality using standard metrics: Variation of Information (VOI), Rand Index (RI), and Segmentation Covering (SC).
Single-view 3D Plane Segmentation. Following PlaneRCNN [16], we generate 3D plane GT labels on the Replica dataset [26] by first fitting planes to the GT mesh using RANSAC [8], and then splitting co-planar regions with different semantics into separate plane instances. total of 498 images are sampled for single-view plane-level segmentation evaluation. Similar to monocular depth estimation, we input two identical images into PLANA3R, and then we merge the output primitives. Tab. 3 and Fig. 4 show that our PLANA3R performs better than the method trained with plane annotation in both plane segmentation and 3D reconstruction. We present more visualization results in the supplementary materials and conduct tests on the 7-Scenes [25] dataset.
Two-view 3D plane segmentation. We also evaluate plane segmentation in the two-view setting. As shown in Fig. 5, our method produces results that are much better than NOPE-SAC. More results are provided in the supplementary materials.
Table 3: Quantitative comparison of single-view planar reconstruction the Replica dataset [26].
The best results are in bold.
Plane Segmentation
Plane Recall (depth)
Plane Recall (normal)
Method
RIf
VOL4
SCT
60.1m
@00.6
615
6300
PlaneRecTR(24]
0.85
1.81
0.58
1.78
13.46
3.79 28.52
18.92
Ours
0.89
1.62
0.63
7.79
30.74
36.31

---

Input Image
PRaneRecTR
Ours
PlaneRecTR
Plane Masks
3D Reconstruction
Figure 4: Comparisons of single-view plane segmentation and 3D reconstruction on the Replica [26].
Input Image
Input Image 2
NOPE-SAC
Ours
Ground Truth
Figure 5: Comparisons of two-view 3D plane segmentation on the Matterport3D [2] (the first two rows) and the S2 [4] (the last row) datasets.
4.4 Multi-view Reconstruction with More Than Two Views
PLANA3R currently supports multi-view reconstruction in pairwise manner, but does not support a single forward pass for inputs with three or more views. Given N input images, we construct N --I image pairs and perform N separate forward passes. The planar primitives predicted from each pair are then merged into common coordinate system. To evaluate this capability, we tested PLANA3R on 50 eight-view samples, sampled every 20 frames from the ScanNerV2 dataset. For fair comparison, we employed MASt3R also in a pairwise manner as the baseline. The quantitative results of the estimated camera trajectories are summarized in Tab. 4. We provide an example of eight-view reconstruction in Fig. 6.
4.5 Alation Study
As introduced in Sec. 3.2 and illustrated in Appendix A.4, to reduce redundancy and achieve more compact and efficient representation, we design the HPPA module, which regulates the number of planar primitives used for rendering and merging by adjusting the gradient threshold 9t- In this analysis, we vary the threshold gta such that, for an input resolution of 512 x 384, the number of generated primitives ranges from 768 (minimum, 512
384 to 3072 (maximum, 512 384
16
By adjusting It, we analyze how the number of pir-view primitives affects two-view 3D reconstruction and single-view depth estimation on ScanNetV2 and NYUv2, respectively. As shown Tab. 5, using approximately half the number bigh-resolution primitives achieves performance comparable to using the full high-resolution set. In contrast, relying solely on low-resolution primitives results in a significant drop in accuracy. These results demonstrate that our method can represent scenes efficiently and compactly using highly sparse set of planar primitives.

---

Table 4: Comparison with MAS(3R on multi-view RRA (Relative Rotation Accuracy) and RTA (Relative Translation Accuracy).
Method RRA@5
@TA@5
RRA@ 10
RTA@ @ 10
RRA@I 15
RTA 15
Ours 0.9000
0.3935
0.9985 0.9964
0.7442 0.5371
1.0000 1.0000
0.8614 0.6878
MAS33
0.9828
0.2657
Figure 6: Reconstruction example from 8 frames of a ScanNet sequence. We show the input 8 frames and planar 3D reconstruction.
Table 5: Ablation study on the gradient threshold (9t.). We show the relationship between the number of per-view primitives and performance.
ScanNetV2 Reconstruction
NYUv2 Depth Estimation
Metrics
Chamfer
F-scoret
Avg. # primitives
RMSEL
51t
Avg. primitives
Ours (0)
0.10
93.10
3072
0.45
86.8
3072
Ours (0.5)
0.11
92.52
1417
0.46
86.4
1565
Ours (10)
0.11
92.32
768
0.49
85.3
768
Conclusion
We present PLANA3R, a zero-shot model for metric planar 3D reconstruction that exploits the geometric regularity of indoor scenes using compact 3D planar primitives. Trained without plane annotations, PLANA3R learns geometry priors from large-scale datasets through transformer-based feature extraction and differentiable planar splatting, relying only on depth and normal supervision. Extensive experiments show that PLANA3R generalizes well oft-oa-domain indoor scenes and supports efficient planar reconstruction, depth estimation, relative pose estimation, and instance-level plane segmentation from unposed image pairs. We believe that our method establishes a solid foundation for large-scale learning of 3D geometry in indoor scenes and holds great potential for advancing the understanding of indoor environments, with broad applicability emerging fields such AR/VR and robotics.
10

---

Acknowledgments
This work was supported by Ant Group Research Intern Program and Ant Group Postdoctoral Pro- gram.
References
[1] S. Agarwala, L. Jin, C. Rockwell, and D. F. Fouhey. Planeformers: From sparse view planes to reconstruction. Eur. Conf. Comput. Vis., volume 13663, pages 192-209 2022. doi: 10.1007/ 978-2-001-20062-55
7,8
[2] A. Chang, A. Dai, T. Funkhouser, M. Halber, M. Niebner, M. Savva, S. Song, A. Zeng, and Y. Zhang. Matterport3d: Learning from rgb-d data indoor environments. In 2017 International Conference on 3D Vision (3DV), pages 667-676 676. JEEE Computer Society, 2017. 3, 6, 7, 23
[3] Z. Chen, Q. Yan, H. Zhan, C. Cai, X. Xu, Y. Huang. W. Wang. Z. Feng. Y. Xu, and L. Liu. Planamerf: Online learning of planar primitives with neural radiance fields. 2025 IEEE International Conference on Robotics and Automation (ICRA), pages 13942 13949. IEEE, 2025. 2
[4] A. Dai, A. X. Chang, M. Savva, M. Halber, T. A. Funkhouser, and M. Nießner. Scannet: Richly- annottaed reconstructions of indoor scenes, IEEE Conf. Comput. Vis. Pattern Recog., pages 2432-2443, 2017.
2, 3, 6, 7, 9, 23, 24
[5] A. Dehghan, G. Baruch, Z. Chen, Y. Feigin, P. Fu, T. Gehauer, D. Kurz, T. Dimry, B. Joffe, A. Schwartz, and E. Shulman. Arkitscenes: A diverse real-world dataset for 3d indoor scene understanding using mobile RGB-D data. In Adv. Neural Inform. Process. Syst., 2021. 2, 6, 24
[6] S. Dong, S. Wang, Liu, L.. Cai, Fan, J. Kannala, and Y. Yang. Reloc3r: Large-scale training of relative camera pose regression for generalizable, fast, and accurate visual localization. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 167:9-167522 2025. 3,4
[7] A. Dosovitskiy, Beyer, A. Kolesnikov, D. Weissenborn, Zhai, T. Unterthiner, Dehghhnn, Minderer, G. Heigold, S. Gelly, Uszkoreit, and N. Houlshy. An image worth 16x16 words: Transformers for image recognition at scale. In Int. Conf. Learn. Represent., 2021. 2, 3,4
[8] M. A. Fischler and R. C. Bolles. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6):381-3955 1981. 8
[9] Y. He, W. Zhao, S. Liu, Y. Hu, Y. Bai, Y. Wen, and Y. Liu. Alphatablets: A generic plane representation for
3d planar reconstruction from monocular vidcos. In Adv. Neural Inform. Process. Syst., 2024. 2,
[10] M. Hu, W. Yin, C. Zhang. Z. Cai, X. Long, H. Chen, K. Wang, G. Yu, C. Shen, and S. Shen. Metric3d v2: A versatile monocular geometric foundation model for zero-shot metric depth and surface normal estimation. IEEE Transactions on Puttern Analysis and Machine Intelligence, 2024. 2,6
[11] L. Jin, S. Qian, A. Owens, and D. F. Fouhey. Planar surface reconstruction from sparse views. In Inf. Conf. Comput. Vis., pages 12971-12980, 2021. doi: 10. (WD9/CCCC44922.2221..00
7,8
[12] B. Ke, A. Obukhov, S. Huang, N. Metzger, R. C. Daudt, and K. Schindler, Repurposing diffusion- based image generators for monocular depth estimation. In IEEE Conf. Comput. Vis. Pattern Recog. 2024. 2
[13] B. Kerbl, G. Kopanas, Leimkühher, and G. Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Trans. Graph., 42(4):139: 13:::-19111111 139:14, 2023. doi: 10. (ol1145//59243... 4
[14] V. Leroy, Y. Cabon, and Revaud. Grounding image matching in with mast3r. In European Conference
on Computer Vision, pages 71-91. Springer, 2024. 2, 3, 7, 8
[15] C. Liu, Yang, D. Ceylan, Yumer, and Y. Furukawa. Planenet: Piece-wise planar reconstruction from single rgb image. In IEEE Conf. Comput. Vis. Pattern Recog., pages 2579-2588, 2018.
[16] C. Liu, K. Kim, J. Gu, Y. Furukawa, and J. Kautz, Plancrenn: 3d plane detection and reconstruction from single image. IEEE Conf. Comput. Vis. Pattern Recog,, pages 4450-4459, 2019. 2, 3,
[17] J. Liu, P. Ji, N. Bansal, C. Cai, Q. Yan, X. Huang, and Y. Xu. Planemvs: 3d plane reconstruction from multi-view stereo. In IEEE Conf. Comput. Vis. Pattern Recog pages 8655-8665, 2022. 3
[18] J. Liu, R. Yu, S. Chen, S. X. Huang, and H. Guo. Towards in fn-the-wild 3d plane reconstruction from a single image. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 27027-27033 2025. 2

---

[19] I. Loshchilov and F. Hutter. Decoupled weight decay regularization. arXiv preprint arXiv: 2017. 6
[20] P. K. Nathan Silberman, Derek Hoiem and R. Fergus. Indoor segmentation and support inference from rgbd images. In ECCV, pages 746-760, 2012. 6,
[21] R. Ranftl, A. Bochkovskiy, and V. Koltun. Vision transformers for dense prediction. Int. Conf. Comput. Vis., pages 12179-12188, 2021. 2,
[22] R. Ranftl, K. Lasinger, D. Hafner, K. Schindler, and V. Koltun. Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer. JEEE Trans. Pattern Anal. Mach. Intell., 44(3): 1623-1637, 2022.
[23] M. Savva, A. Kadian, O. Maksymets, Y. Zhao, E. Wijmans, B. Jain, J. Straub, J. Liu, V. Koltun, J. Malik, et al. Habitat: A platform for embodied research. IEEE Conf. Comput. Vis. Pattern Recog,, pages 9339-9347, 2019. 2, 6, 21,24 24
[24] J. Shi, S. Zhi, and K. Xu. Plamerectr: Unified query learning for 3d plane recovery from single view. In Inf. Conf. Comput. Vis., pages 9343-9352, 2023. 7. 7,8
[25] J. Shotton, B. Glocker, C. 7ach, S. Tzadi, A. Criminisi, and A. Fitzgibhon. Scene coordinate regression forests for camera relocalization in rgh-d images. In Int. Conf. Comput. pages 2930-2937, 2013. 6,
21,
[26] J. Straub, T. Whelan, L. Ma, Y. Chen, E. Wijmans, S. Green, J. J. Engel, R. Mur-Artal, C. Ren, S. Verma, A. Clarkson, M. Yan, B. Budge, Y. Yan, X. Pan, J. Yon, Y. Zou, Leon, N. Carter, J. Briales, T. Gillingham, E. Mueggler, Pesqueira, M. Savva, D. Batra, H. M. Strasdat, R. Nardi, M. Goesele, S. Lovegrove, and R. Newcombe. The Replica dataset: A digital replica of indoor spaces. arXiv preprint
2019. 6, 8, 2,
[27] B. Tan, Xue, S. Bai, Wu, and Xia. Planetr: Structure guided transformers for 3d plane recovery. In Int. Conf. Comput. Vis., pages 4166 4175, 2021. 3, 8
[28] B. Tan, N. Xue, T. Wu, and G. Xia. NOPE-SAC: neural one-plane RANSAC for sparse-view planar 3d reconstruction. IEEE Trans. Puttern Anal. Mach. Intell., 45(12):15233-15245 2023, doi: 10.1 109T/AAA.
2023.3314745. 2, 2,3,7,8 7,8
[29] B. Tan, R. Yu, Y. Shen, and Xue. Planarsplatting: Accurate planar surface reconstruction in minutes. In Proceedings of Computer Vision and Pattern Recognition Conference, pages 1190-1199, 2025. 2, 3,
4,5
[30] Z. Tang, Fan, D. Wang, H. Xu, Ranjan. A. Schwing, and Z. Yan. Mv-dust3r+: Single-stage scene reconstruction from sparse views in 2 seconds. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 5283-5293, 2025. 2,
[31] H. Wang and L. Agapito. 3d reconstruction with spatial memory. In 2025 International Conference on 3D Vision (3DV), pages 78-89, IEEE, 2025. 2.
[32] J. Wang, M. Chen, N. Karaev, A. Vedaldi, C. Rupprecht, and D. Novotny. Vggt: Visual geometry grounded transformer. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages
2025. 2,
[33] S. Wang, V. Leroy, Y. Cabon, B. Chidlovskii, and J. Revaud. Dust3r: Geometric 3d vision made easy. In IEEE Conf. Comput. Vis. Pottern Recog., pages 20697-207777 207009, 2024. 3,
[34] J. Watson, F. Aleotti, M. Sayed, Z. Qureshi, O. M. Aodha, G. J. Brostow, M. Firman, and S. Vicente. Airplanes: Accurate plane estimation via 3d-consistent embeddings. In IEEE Conf. Comput. Vis. Pattern Recog.. pages 3270-5280 2024. 3
[35] P. Weinzaepfel, V. Leroy, T. Lucas, R. Brégier, Y. Cabon, V. Arora, L. Antsfeld, B. Chidlovskii, G. Csurka, and J. Revaud. Croco: Self-superviedd pre-training for 3d vision tasks by cross-view completion. Adv. Neural Inform. Process. Syst., 2022,
[36] Y. Xie, M. Gadelha, F. Yang. X. Zhou, and H. Jiang. Planarrecon: Realtime 3d plane detection and reconstruction from posed monocular videos. In IEEE Conf. Comput. Vis. Puttern Recog., pages 6209-.. 6218, 2022. 2,
[37] J. Yang. A. Sax, K. J. Liang. M. Henaff, H. Tang, A. Cao, J. Chai, F. Meier, and M. Feiszli. Fast3r: Towards 3d reconstruction of 1000+ images in one forward pass. In Proceedings of the Computer Vision and Puttern Recognition Conference, pages 21924 21924-21935, 2025. 2,
12

---

[38] H. Ye, Y. Liu. Y. Liu, and S. Shen. Neuralplane: Structured 3d reconstruction in planar primitives with neural fields. In Int. Conf. Learn. Represent., 2025. 2,
[39] C. Yeshwanth, Y. Liu, M. Nießner, and A. Dai. Scannet++: A high-fidelity dataset of 3d indoor scenes, In Int. Conf. Comput. Vis., pages 12 12-22, 2023. 6, 24
[40] Z. Yu, J. Zheng. D. Lian, Z. Zhou, and S. Gao. Single-image piece-wise planar 3d reconstruction via associative embedding. In IEEE Conf. Comput. Vis. Puttern Recog, pages 1029-1037, 2019. 3,
[41] S. J. J. Wang, Y. Xu, N. Xue, C. Rupprecht, X. 7hou, Shen, and G. Wetzstein. Flare: Feed-forwan forword geometry. appearance and camera estimation from uncalibrated sparse views. In Proceedings of the Computer Vision and Puttern Recognition Conference, pages 21936-21947, 2025. 2,
[42] W. Zhao, Liu, S. Zhang, Y. Li, Chen, X. Huang, Y.-. Liu, and H. Guo. Monoplane: exploiting monocular geometric cues for generalizable plane reconstruction. 2024 JEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 8481-8488. JEEE, 2024. 2
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
See our Abstract and Sec.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: See Appendix A.5.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
14

---

Justification:
This paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced
statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Question:
Does the paper fully disclose all the information needed reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We have provided detailed descriptions of our network architecture, the training and testing datasets, the evaluation protocols, and the implementation details of our experiments. Comparisons with existing methods conducted using their official open-source implementations.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, a No answer this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing verifying the results.
5. Open access to data and code
15

---

Question: Does the paper provide open access to the data and code, with sufficient instructions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: All the datasets used in this paper are public. The code is publicly available on our project page at :ttps://1c2666666 github io/plana3r/.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips cc/ wttpc//guiees//odeeeee
oeoaenissssoooloiiioooioommoooee olicy) for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: / //nips. cc/ ctc/public/guuti http//guiiee/eeeee geguidde//ooeuubnnn
cessinnoolccnn for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question:
Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Please check Sec. 4.1 and Appendix A.2.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer:
[Yes]
Justification:
The statistical significance of the experiments is provided the paper.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" the results are accompanied by error bars, confidence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
16

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma enror bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Answer: [Yes]
Justification: Please check Sec. 4.1 and Appendix A.3.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics btttps//neurr: cc httpppulll/ttiicceoeee csCuidel ines?
Answer: [Yes]
Justification:
The research conducted in the paper conforms with the NeurIPS Code of
ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consideration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The paper has discussed both the potential positive societal impacts and negative societal impacts of the work performed.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
17

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path any negative applications, the authors should point it out. For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility MI).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: Our model was trained on publicly available datasets. We did not observe any risk for misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We have correctly cited the references regarding to the public datasets and models. Our usage did not violate the license and terms of use of those assets.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
18

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. cww datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Question:
Answer: [Yes]
Justification: All code and models for our proposed method are currently well-documented in a private repository. Upon public release, we will provide comprehensive documentation to support replication and usage, along with the associated assets.
Guidelines:
The answer NA means that the paper does not release assets.
Researchers should communicate the details the dataset/code/moodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsoureing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification:
This paper does involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
19

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: LLM is used only for writing, editing, or formatting purposes this paper.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https: :thppa:/nnee wttp///een...peeeeeee cc/ wwtpa//ccc//e/ecc/////// for what should or should not be described.
20

---

A Technical Appendices and Supplementary Material
This appendix provides more details and results of our PLANA3R.
A.1 Extra Results
We present additional visualization results here.
Single-view 3D Reconstruction and Plane Segmentation. To further demonstrate the zero- shot generalization capability of our method in out-of-domain scenes, evaluate single-view reconstruction and planar segmentation on the 7-Scenes dataset [25]. Although the 7-Scenes dataset widely used indoor dataset and is very suitable for out-of-domain evaluation, does not provide official plane segmentation masks. We attempted to generate them via mesh synthesis but found it challenging obtain high-quality ground truth labels. Therefore, present the visualization results. As shown in Fig. 7, our method achieves much better planar segmentation and reconstruction performance than PlaneRecTR..
Two-view 3D Reconstruction and Plane Segmentation. We provide more results in Fig. 8 to show that PLANA3R achieves much better performance than NOPE-SAC.
A.2 Implementation Details
Data Augmentation. PLANA3R is trained on combination of four public indoor datasets as shown in Tab. 7. Due to the low visual quality of rendered RGB images in Habitat [23], include only small subset of synthetic images during training and primarily rely on real-world data. Following the training strategy DUSt3R [33], we apply color jittering and construct symmetrie image pairs. (I* 12) and (12, 1), for data augmentation. In our training set (totally around 4M image pairs), include approximately 0.57M image pairs with overlap, while the remaining 3.43M pairs are randomly sampled from nearby frames (mainly within the next 10 frames). We observed that incorporating the 0.57M non-overlapping pairs helps the model learn strong geometric priors, enabling it to infer relative pose and scene geometry even when the two views have little or no overlap, as shown in Fig. 8.
Table 6: Results on ScanNetV2 and MP3D (w/o Ext. Data means training without 0.57M non- overlapping image pairs).
Translation Mean (m))
Rotation Mean (*) $10
Overlap
Training Data
Med. (m)
$0.2mf
Med. (P)
ScanNetV2
Full
0.05
0.07
97.40
1.53
1.80
99.72
Easy (>50%)
w/o Ext. Data
0.05
0.09
95.73
1.77
2.77
99.16
Full
0.06
0.10
93.56
2.02
2.68
98.88
Medium (15-50%)
Data
0.07
89.68
2.25
4,24
97.60
Full
0.12
0.26
71.98
2.68
5.53
95.67
Hard (<15%)
w/o Ext. Data
0.15
0.41
63.03
3.46
10.35
88.30
Full
0.19
0.37
51.26
2.87
7.45
94.12
Very Hard (non-overlap)
w/o Ext. Data
0.23
0,53
46.22
4.16
16.64
79.83
Full
0.07
0.13
89.16
2.01
3.16
98.30
All
w/o Ext. Data
0.07
0.20
84.60
2.31
5.38
95.68
MP3D
Full
0.20
0.33
49.72
1.65
2.37
99.15
Easy (>50%)
w/o Ext. Data
0.23
0.49
44.84
1.92
4.29
97.08
Full
0.25
0.45
37.55
2.19
4.46
96.41
Medium (15-50%)
w/o Ext. Data
0.29
0.66
36.75
2.41
7.67
91.81
Hard (<15%)
Full
0.46
0.82
17.35
2.96
11.12
85.97
Ext.
Data
0.59
14.26
3.17
17.84
79.52
Very Hard (non-overlap)
Full
0.82
1.11
5.63
3.00
12.37
85.28
w/o Ext. Data
0.85
1.11
5.19
2.98
18.84
79.65
Full
0.24
0.45
40.16
2.00
4.49
96.12
woo Ext. Data
0.29
0.64
37.14
2.24
7.62
92.37
21

---

Input Image
Ours
PlaneRocTR
Ours
PlaneRecTR
Plane Masks
3D Reconstruction
Figure 7: Supplement comparisons of single-view plane segmentaion and 3D reconstruction on the 7-Scenes [25] (the first four rows) and Replica [26] (the last four rows).
Performance Analysis on Pair's Overlap. We also conduct an additional ablation study to evaluate the impact of incorporating the 0.57M non-overlapping image pairs on model performance during training. addition, we analyze how the overlap ratio affects performance. Specifically, we define the overlap ratio as the maximum percentage of pixels in one image (either the first the second) that have direct correspondences the other. We provide quantitative analysis to show the impact of including the 0.57M non-overlapping image pairs in Tab. 6.
22

---

Input Image!
Input Image 2
NOPE-SAC
Ours
Ground Truth
Figure 8: Supplement comparisons of two-view 3D plane segmentaion on the ScanNetV2 [4] (the first rows) and Matterport3D [2] (the last three rows).
Input Image
Input Image 2
NOPE-SAC
Ours
Ground Truth
Figure 9: Supplement comparisons of two-view 3D plane reconstruction on the ScanNetV2 [4] (the first two rows) and Matterport3D [2] (the last three rows).
As shown in Fig. 11 and Tab. 6, including these non-overlapping pairs improves the model's 's overall performance. Furthermore, we observe that as the overlap ratio the test set decreases, the model's accuracy consistently degrades.
23

---

Figure 10: Reconstruction example from 8 frames of a ScanNet sequence. We show the input frames and planar 3D reconstruction.
Performance vs Overtap (ScanNetV2)
Performance Overlap (MP3D)
90
92-
80
60
10
40
80
Tans <m1.. (wnyFatttinal
Tens <m0.2m
20
Thans ied 7m (enyfattaane)
Trens 2m (ful)
50
Got c-10" c-10"(full
Fat 1-10" gfull
(mstEstDeta)
(vorExtDalal
Easy (07))))
Medium (15-50%)
Hard (c19%))
Very Horld
Easy (+-005))
Medium (15-50%)
Hard (cd((
Very Hord
Figure 11: Test-time performance versus overlap degree.
Plane merge. Our PLANA3R predicts a set of 3D planar primitives from the inputs. To further achieve instances of 3D planes, we merge the predicted primitives according their spatial connectivity and geometric similarity. In detail, we greedily merge adjacent planar primitives that meet the merging thresholds (projection distance 0.1m and normal angle difference S 25 degrees) to obtain the final 3D plane instances.
Table 7: Training data and sample sizes for PLANA3R.
Datasets
Type
N Pairs
ScanNetV2
Indoor/Real
610K
ScanNet++ [39]
Indoor/Real
810K
ARKitScenes [5]
Indoor/Real
2400K
Habitat [23]
Indoor/Symthetic
120k
For our final model used for evaluation, we set the loss weights a1 5, 02 5. 0g 20 in Eq. (3). We set the loss weights 31 1, 82 1, A8g Eq. (4). We set the loss
Hyperparameters.
weights 71 10, 72 10, 73 = 1 in Eq. (5). We observed that increasing the weights of the patch

---

loss facilitates faster convergence during the training process. This improvement can be attributed to the fact that more accurate primitive depth estimation ensures proper differentiable rendering and gradient propagation for larger number of primitives. This occurs because primitives with very incorrect spatial positions or orientations lie outside the camera's viewing frustum, preventing them from being rendered.
A.3 Runtime Analysis
We evaluate the inference runtime of our PLANA3R using an NVIDIA RTX 3090 GPU. On average. single feed-forward pass takes 70 ms for predicting planar primitives and relative camera pose. Efficient CUDA-based planar primitive rendering achieves rate of 1000 fps.
A.4 Planar Primitive and Primitive Selection
We show the representation of 3D planar primitive in Fig. 12 (a). We further explain the module of primitive selection in Fig. 12 (b).
High reselution petches
Rendered normal/dapph maps
Low www.cssolutionn
1/
H
HxW
16
W B
(a) 3D planar primitive
(b):Primitive selection
HxW
Figure 12: (a) 3D planar primitive using learnable shape parameters. V and V are the positive direction of the X-axis and Y-axis, respectively. (b) We compute the gradient magnitude for each pixel in x W low-resolution predicted normal patches Npatch To combine the low- and high-resolution primitives, we use binary masks (with white indicating valid regions) to merge only the valid patches from both resolutions, rather than directly using all predicted primitives.
A.5 Limitations
We conduct extensive experiments on five test sets. However, when evaluating plane segmentation, the absence high-quality out-of-domain datasets with reliable plane-level annotations poses challenge. Although we attempt to generate pseudo-labels, their limited accuracy restricts us to primarily qualitative evaluation. While this represents limitation our current analysis, also highlights the urgent need for better benchmarks in this field. Despite this, the visual results demonstrate the strong potential of our zero-shot method producing accurate pseudo-plat plane annotations.
A.6 Social Impacts
Positive Social Impact. Our zero-shot, pose-free planar primitive framework significantly broadens the applicability of planar 3D reconstruction. Due its lightweight design and lack of reliance on camera pose or plane annotations, our model can be casily deployed in real-world applications such as AR/VR and robotics, enhancing 3D scene understanding and perception in indoor environments.
Potential Negative Impact. The ease of deployment of our pose-free, zero-shot planar reconstruc- tion model raises potential privacy concerns. Its ability to reconstruct 3D indoor scenes from sparse image pairs could be misused to capture private environments without consent, such as personal residences. While our model performs robustly across various datasets, rare failure cases may occur in challenging scenarios. In real-world deployments, integrating our model into more comprehensive system necessary filter out occasional noisy predictions.
25


Reviews:
[{"review_id": "k1XJ5O8JO8", "reviewer": "Reviewer_9yaB", "summary": "This paper proposes PLANA3R, a two-view metric planar reconstruction method designed for indoor environments. Built upon the DUSt3R network framework, the authors introduce network heads that output multi-level planar primitives and utilize normal gradients to guide the selection of primitives, thereby reducing the number of primitives. Additionally, the method leverages normal and depth images from the dataset to supervise training, without requiring ground-truth plane annotations. Experimental results demonstrate that PLANA3R outperforms the baselines on tasks such as 3D surface reconstruction, depth estimation, and relative pose estimation.", "questions": "see Weaknesses.", "limitations": "Yes, in the supplementary.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths:**\n1. The motivation of the paper is clear: planar primitives are well-suited for representing lightweight, structured indoor environments. The Introduction and Related Work sections are logically organized and easy to follow. \n2. The use of multi-level, multi-resolution primitive outputs combined with a normal gradient-based selection mechanism effectively reduces the number of primitives.\n3. Experimental results demonstrate that the proposed method delivers strong performance across multiple tasks.\n\n**Weaknesses:**\n1. Since PlanarSplatting is a fundamental component of this work, it would be helpful to include a more detailed explanation of its core ideas to make the paper more self-contained and accessible to readers unfamiliar with it.\n2. The Method section lacks clarity in several important aspects, making it difficult to follow. For example, in Equation (1), the meaning of the indices $i$ and $j$ in $d_{\\pi}^{i,j}$, $\\mathbf{r}{\\pi}^{i,j}$, and $\\mathbf{q}{\\pi}^{i,j}$ is not clearly explained. Does $\\mathbf{q}{\\pi}^{i,j}$ represent the quaternion of a primitive generated from image $i$, expressed in the coordinate frame of camera $j$? or $\\mathbf{q}{\\pi}^{i,j}$ is simply under its camera coordinate system.  The lack of clarity regarding the coordinate systems involved makes it difficult to fully understand how the training losses are formulated and applied. \n3. In the *Training losses and training strategies* section, \"As a result, directly applying the differentiable planar primitive rendering .......\"\nActually, I don't understand what the rendering is used for. I couldn't find any information that explained it. Do you mean render the depth and normal images from image $i$ to camera $j$, or render images from all the primitives under the same coordinate system (use $q$)? And then use the rendered image to compare it with the ground truth? What's the target for introducing this rendering loss to the training stage? \n4. It is also not clear whether the warm-up training phase includes the pose loss. \n\nOverall, I appreciate the value of this work and its strong performance across different tasks. However, the lack of clarity in important implementation details makes it difficult to fully understand how the proposed method is realized. If the authors can provide more detailed explanations regarding these aspects, I would be willing to consider increasing my score.", "was_revised": true, "final_justification": "The authors have responded to my concerns appropriately and professionally. Given the impressive results demonstrated in this work, I have decided to raise my score to Borderline accept.", "rebuttal": "We sincerely thank you for your thorough and insightful feedback, which has significantly enhanced the quality of our manuscript. We greatly appreciate the recognition of the value and performance of our approach. Below, we address your suggestions and questions with detailed clarifications and outline revisions.\n\n### 1. On PlanarSplatting introduction\nThank you very much for your suggestion. We will include an introduction to PlanarSplatting in our revised manuscript to improve clarity and make the paper more self-contained. Specifically, we will add the following description in Section 3.1:\n\n>'' PlanarSplatting's rasterization is a core component of our Plana3r model. PlanarSplatting is an ultra-fast and accurate method for reconstructing planar surfaces in indoor scenes from multi-view images. Rather than detecting, matching, or tracking planes in 2D or 3D, it directly operates on a set of 3D planar primitives, \"splatting\" them into **dense depth and normal maps** via differentiable, CUDA-accelerated rasterization.''\n\n### 2. Symbol definitions and coordinate frames\n\nWe apologize for the insufficient explanation of symbols in the manuscript. Your interpretation of $q_{\\pi}^{i,j}$ is correct—it denotes the quaternion of a planar primitive generated from image $I^i$, represented in the coordinate frame of camera $j$. In our training setup, **we fix $j = 1$, treating $I^1$ as the reference frame**. Accordingly, the ground-truth camera pose of $I^1$ is set to the $4{\\times}4$ identity matrix.  The notation $r_{\\pi}^{i,j}$ refers to the radii array of planar primitives associated with image $I^i$. Since the radii represent side lengths and are independent of camera $j$, this notation may be misleading. We will revise it to $r_{\\pi}^{i}$ for clarity. Similarly, $d_{\\pi}^{i,j}$ was a typo and should be corrected to $d_{\\pi}^{i}$, which denotes the depth map of the planar primitive centers associated with image $I^i$. We will revise the Equation (1) in manuscript to clarify the notation and coordinate system accordingly as:\n> Input: \\$(I^i, K^i)_{i=1,2}\\$\n\n> Output of Plana3r: $(d_{\\pi}^{i},\\ r_{\\pi}^{i},\\ q_{\\pi}^{i,j})$, $j{=}1$, and $P_{\\text{rel}}$.\n\nThen, the primitive centers $c_{\\pi}$ from both $I^1$ and $I^2$ are transformed into the coordinate frame of camera $j=1$ using the corresponding intrinsics $K^i$, depthmap $d_{\\pi}^{i}$ and camera poses (page 4, line 127 to line 128). Therefore, throughout training, the primitive centers $c_{\\pi}$, quaternions, and camera poses of both $I^1$ and $I^2$ are consistently expressed in the coordinate frame of camera $j = 1$. \n\n### 3. Differentiable planar primitive rendering in training stage\n\nWe appreciate the reviewer’s question regarding the phrase: *“As a result, directly applying the differentiable planar primitive rendering...”* Here we provide further clarification:\n\n **- What to render:**\n\nOur **Plana3r** model takes as input two unposed images and their camera intrinsics, and outputs a set of planar primitives in the coordinate frame of the camera \\$j{=}1\\$. During training, we use the differentiable rasterizer from PlanarSplatting to render dense depth and normal maps **at the original resolution (\\$H \\times W\\$)** with sparse primitives predicted from low-resolution image features (Page 15, Figure 9 (b)).\n\n* For \\$I^1\\$, rendering is done using the \\$4{\\times}4\\$ identity matrix as its camera pose.\n* For \\$I^2\\$, rendering is done using the ground-truth relative pose from \\$I^2\\$ to \\$I^1\\$.\n\nThe rendered depth \\$D^{\\text{render}}\\$ and normal \\$N^{\\text{render}}\\$ for each image are computed in their respective camera coordinate systems. Thus, while the inputs to the renderer (camera poses and primitives) are defined in the coordinate frame of \\$I^1\\$, the rendered outputs (depth and normal maps) are expressed in each image’s own coordinate frame.\n\n**- Target of rendering loss:**\n\nNote that the primitives are initially predicted from low-resolution image patches and are supervised using patch-level losses at resolutions of ($H/16 \\times W/16$) and ($H/8 \\times W/8$), as described in Equation (3).  **This patch-level loss does not use the radii information of planar primitives, but only supervises the depth and quaternion of predicted planar primitives**. To capture accurate geometry of the scene and let the gradient flow optimize the radii of planar primitives, we further supervise the model by computing losses between the rendered and ground-truth depth and normal maps (as shown in Equation (4)) using PlanarSplatting's rasterization, at the original image size (\\$H \\times W\\$). This encourages the model to **predict accurate primitive shapes (radii)**, depths, and orientations, thereby improving the geometric fidelity of the reconstructed scene. We will incorporate this clarification into the manuscript.\n\n### 4. On warm-up training phase\nThank you for highlighting the need for clarity regarding the warm-up training phase. As noted in Section 4.1 (page 6, line 205 to line 206), the warm-up phase includes the pose loss. We will ensure this is more prominently explained in the revised manuscript to avoid any oversight.\n\nWe hope these clarifications address your questions effectively. Thank you again for your valuable reviews."}, {"review_id": "5VpAgr6As0", "reviewer": "Reviewer_w5iN", "summary": "This paper proposes PLANA3R, a strong method that extends the 3R framework to handle structured indoor scenes. The key idea is to pretrain a feed-forward plane prediction network that learns to generate sparse planar primitives in metric scale, using only depth and normal maps as supervision from large indoor datasets. The differentiable PlanarSplatting module plays a key role for end-to-end training of the pipeline. PLANA3R predicts planar 3D primitives directly from two unposed images, supporting multiple downstream tasks such as depth estimation, plane segmentation, and reconstruction. Extensive experiments demonstrate that PLANA3R achieves impressive performance, showing clear improvements over prior methods in both qualitative visualizations and quantitative metrics.", "questions": "1.\tThere are many incomplete depths in GT datasets because they are obtained from sensored LiDAR cameras. How do you deal with the incompleteness in GT depths?\n2.\tPLANA3R uses estimated normal from Metric3Dv2 in scenarios where there is no GT normal. However, we can also generate pseudo normal from GT depths just as what 2DGS does. It may provide more accurate normal supervision. Does the author observe any difference in quality or training stability between these two pseudo normals?\n3.\tWhy are the input images in Figure 3,4,5 with such low quality? Aren’t they come from GT image sequences?", "limitations": "No evident limitations.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n\n1. This paper is an interesting and meaningful extension of the 3R framework towards structured planar 3D reconstruction. By leveraging PlanarSplatting as a differentiable renderer, the entire training pipeline is made fully end-to-end trainable without explicit plane-level annotations, which is a notable technical strength.\n2. The proposed hierarchical primitive prediction combines low-resolution and high-resolution primitives to achieve a balance between primitive numbers and reconstruction fidelity, which is novel and practically effective.\n3. The experimental setup is thorough, covering diverse datasets and multiple tasks, with strong improvements over baselines in both quantitative metrics and visual quality.\n4. The paper is well-written, the technical presentation is clear, and the results are convincingly demonstrated.\n\nWeaknesses\n\nI did not find any major weaknesses. One minor point: I suggest the author to reorganize Figure 2 to more clearly illustrate how the low-resolution and high-resolution primitives are generated. As currently drawn, it may leave readers with the impression that the high-res primitives are simply upsampled directly from the low-res ones, which is not the case.", "was_revised": true, "final_justification": "All of my concerns have been addressed. I would keep my rating of accept.", "rebuttal": "We sincerely thank you for your positive feedback on our paper. We are especially grateful that you found the research topic intriguing and recognized the novelty and technical strengths of our proposed method. Your insightful comments have helped us refine our work further. Below, we provide detailed responses to your suggestions and questions, along with the revisions we plan to make.\n\n### 1. On framework figure\nWe appreciate your suggestion to enhance Figure 2 and the descriptions of high-resolution and low-resolution primitives. In the revised manuscript, we will update Figure 2 to better illustrate these concepts and provide more detailed explanations of high-resolution and low-resolution primitives in the manuscript to improve clarity and accessibility.\n\n### 2. On sensor ground-truth depth\nWe acknowledge that the sensor-provided ground-truth depth is relatively sparse. During training, we apply masks to filter out invalid values (e.g., NaNs) in the depth data. These masked regions are excluded from the loss computations in Equations (3) and (4).\n\n### 3. On pseudo ground-truth normals\nThank you for raising this insightful point. Following your suggestion, we generated pseudo ground-truth normals from the sensor depth data. Compared to the normals generated by Metric3Dv2, these pseudo normals contain more invalid regions and exhibit less smoothness. To evaluate their impact, we trained a version of our model using these pseudo normals as supervision, as shown in the table below. It can be seen that the model performs better when using the normal provided by Metric3Dv2. Interestingly, using a poor normal not only reduces the performance of 3D reconstruction but also the accuracy of pose estimation.\n\n**ScanNetV2 dataset**\n| Method                        |        Pose Translation        |                      |  Pose Rotation           |                      | Reconstruction |               |\n|------------------------------|--------------------|----------------------|----------------------|----------------------|----------------|---------------|\n|                              | Med. (m)↓           | Mean (m)↓             | Med.(deg)↓            | Mean(deg)↓            | Chamfer ↓      | F-score ↑     |\n| normal_from_depth            | 0.11  | 0.18  | 4.22      | 5.78      | 0.22       | 73.18      |\n| Full (normal from metric3d)  | **0.07**  |  **0.13**  |  **2.01**      |  **3.16**      | **0.11**       |  **92.52**      |\n\n**MP3D dataset**\n\n| Method                        |        Pose Translation       |                      |        Pose Rotation               |                      | Reconstruction |               |\n|------------------------------|--------------------|----------------------|----------------------|----------------------|----------------|---------------|\n|                              | Med. (m)↓           | Mean (m)↓             | Med.(deg)↓            | Mean(deg)↓            | Chamfer ↓      | F-score ↑     |\n| normal_from_depth            | 0.28  | 0.47  | 4.32      | 6.59      | 0.34       | 49.13      |\n| Full (normal from metric3d)  |  **0.24**  |  **0.45**  |  **2.00**     |  **4.49**      |  **0.32**       |  **56.63**      |\n\n### 4. On MP3D RGB images\nThe RGB images in the MP3D dataset are rendered from the officially provided mesh, and thus exhibit lower quality and certain distortions compared to real camera-captured images. Nonetheless, these inputs further underscore the strong generalization capability of our Plana3r model. This result highlights the effectiveness of planar primitives as a robust intermediate representation: despite the degraded image quality, Plana3r consistently predicts high-quality planar primitives, enabling accurate reconstruction of planar surfaces.\n\nWe hope these clarifications and planned revisions effectively address your feedback. Thank you again for your valuable insights, which have significantly strengthened our manuscript."}, {"review_id": "ZIk1FCQExc", "reviewer": "Reviewer_CYmQ", "summary": "The paper proposes PLANA3R, a feed-forward, pose-free framework for metric-scale planar 3D reconstruction from unposed two-view images. It uses sparse planar primitives and learns from depth and normal maps without plane annotations and shows strong zero-shot generalization across indoor datasets.", "questions": "1. Can the model be extended to handle more than two input images? Have you experimented with applying the framework to reconstructions using a different number of input views?\n2. Could the authors provide a quantitative analysis of the computational performance achieved by the hierarchical feature design, including measurements of runtime, memory consumption, or floating point operations (FLOPs)?\n3. How much resource consumption and calculation speed increase does the design of image texture mask bring?\n4. Are there typical patterns or scenarios where the hierarchical design fails, e.g., in low-texture areas or complex clutter?", "limitations": "The authors have adequately addressed the limitations and societal impact in the main paper.", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengthens：\n- The use of planar primitives offers a concise and interpretable alternative to dense point clouds.\n- Extensive experiments demonstrate state-of-the-art performance across different tasks.\n- The approach trains without explicit plane-level supervision, relying only on depth and normal maps, which are more readily available.\n- PLANA3R predicts relative pose and 3D structure in a single forward pass, making it suitable for real-time applications.\n\nWeaknesses：\n- The method feels like a relatively direct combination of ideas from PlanarSplatting and DUSt3R/MASt3R, with limited conceptual novelty.\n- It is unclear how well it generalizes to multi-view or more complex scenes beyond pairwise settings.\n- The hierarchical prediction of 16× and 8× downsampled features constitutes a core design intended to select appropriate fine-grained textures in different regions to reduce computational overhead and improve inference speed. However, the authors have not yet provided quantitative evidence, such as measurements of runtime, memory consumption, or floating-point operations (FLOPs). Moreover, although the design is intended to accelerate computation, it requires computing gradient-based texture masks, which itself incurs additional computational cost. It remains unclear whether this overhead offsets the claimed improvement in computational speed.", "was_revised": true, "final_justification": "Thank authors for their very detailed responses and additional experiments. My concerns have been resolved by the responses. I decide to raise my score accordingly.", "rebuttal": "We sincerely thank the reviewers for recognizing the training strategy and application value of our Plana3r method, which enables end-to-end real-time reconstruction and pose estimation without requiring plane annotations. Below, we address your concerns with detailed clarifications and planned revisions.\n\n### 1. On the Conceptual Novelty of Plana3r\n\nThe feedforward modeling of multi-view geometry (MVG) gained traction with the introduction of Dust3r/Mast3r in early 2024, followed by other “3R” family methods addressing longstanding challenges in classical MVG through point map representations and large vision transformers, including but not limited to multi-view aggregation (VGGT [1]) and dynamic scene reconstruction (Cut3r [2]). While these methods have improved 3D reconstruction performance, **point map representations are often redundant and inefficient**. All existing 3R methods must output a large number of 3D points to characterize a scene. Thus, pursuing compact and efficient representations in feedforward 3D reconstruction is both urgent and impactful.\n\nPlana3r addresses this by introducing a compact planar 3D representation into the 3R framework, integrating the strengths of feedforward modeling and planar primitives. We build upon PlanarSplatting, which provides an efficient rendering scheme from planar primitives. However, unlike PlanarSplatting, our method does not require precomputed camera poses. From that perspective, Plana3r is a non-trivial technical advancement. Our key contributions are as follows:\n\n* **Sparse Planar Representation**: Unlike Dust3r/Mast3r, which use a DPT head for dense point regression, and PlanarSplatting, which requires dense depth maps for primitive initialization, Plana3r directly regresses sparse planar primitives from \\$H/16 \\times W/16\\$ patches. Our novel HPPA module adaptively adjusts the number of primitives, ensuring a sparse yet sufficient representation that captures scene geometry. This reduces storage demands while preserving detail in textureless regions. **Devising a new 3D vision foundation model using the planar representation has unique application value**.\n\n* **Metric Reconstruction and Pose Estimation**: Unlike most related work, Plana3r supports metric reconstruction and pose estimation. Leveraging the good data distribution of indoor data, our model achieves high data utilization and accurate metric estimation, offering new insights into 3D vision foundation model design.\n\n* **Optimized Training Strategy**: Training with differentiable rendering in PlanarSplatting presents challenges due to the randomness in initialization at the start of training—many primitives are initially placed outside the camera’s view, leading to unstable gradients. To mitigate this, we adopt a two-phase training strategy designed to stabilize the training.\n\n### 2. On the motivation of Hierarchical Primitive Prediction Architecture (HPPA)\nWe believe there may be a slight misunderstanding regarding the motivation behind **HPPA**. As stated in our paper (Section 3.1, line 143 to line 145, Page 4): \n> *“To balance efficiency and representational fidelity, we propose a hierarchical primitive prediction architecture (HPPA) to fit the scene using planar primitives, enabling compact and efficient modeling of scene geometry with sparse primitives.”* \n\nOur primary goal with HPPA is to achieve **a more compact and efficient geometric representation using fewer primitives**. Here, the term **\"efficiency\"** specifically refers to the **number of primitives required to represent a scene**, rather than the inference runtime. While we do not claim that HPPA improves inference speed, we do provide a runtime and FLOPs breakdown to demonstrate that the **additional computational overhead introduced by the image texture mask is minimal**. As the calculations are conducted on $H/16 \\times W/16$ and $H/8 \\times W/8$ patches, the extra cost is negligible in practice. If the term **“efficiency”** has caused any confusion, we will revise the wording in **Section 3.1 and Section 3.2** to clarify that it refers to **representational efficiency (i.e., fewer primitives)** rather than runtime efficiency. Below is the detailed runtime analysis tested on NVIDIA RTX 3090:\n\n```\nencoder:               12.1 ms  \ndecoder:               32.8 ms  \nupsample:               0.4 ms  \nhead_prediction:       18.0 ms  \nhppa_low_view1:         2.0 ms  \nhppa_high_view1:        1.5 ms  \nhppa_low_view2:         1.0 ms  \nhppa_high_view2:        0.9 ms  \nrendering:              0.9 ms  \n-------------------------------\nTotal HPPA overhead:    around 5.0 ms  \nTotal compute:         ~1.54 TFLOPs  \nMemory footprint:      ~7.8 GB  \nModel params: 540M (DUSt3R is 571M, MASt3R is 689M)\n```\n\n### 3. On the multi-view  reconstruction with more than two input images\nPlana3r currently supports multi-view reconstruction in a pairwise manner, but does not support a single forward pass for inputs with three or more views. Given $N$ input images, we construct $N-1$ image pairs and perform $N-1$ separate forward passes. The planar primitives predicted from each pair are then merged into a common coordinate system. To evaluate this capability, we tested Plana3r on 50 eight-view samples, sampled every 20 frames from the ScanNetV2 dataset. For fair comparison, we employed MASt3R also in a pairwise manner as the baseline. The quantitative results of the estimated camera trajectories are summarized below:\n\nRRA (Relative Rotation Accuracy) and RTA (Relative Translation Accuracy)\n| Metric  ↑ | Ours    | mast3r |\n|----------|--------|-----------|\n| RRA@5     | 0.9000 |  **0.9828**    |\n| RTA@5    | **0.3935** | 0.2657    |\n| RRA@10   | **0.9985** |  0.9964    |\n| RTA@10   | **0.7442** | 0.5371    |\n| RRA@15   | **1.0000** | **1.0000**    |\n| RTA@15   | **0.8614**    | 0.6878    |\n\nThis paper focuses on two-view reconstruction and relative pose estimation—core problems in 3D vision with broad applicability. Our key contribution lies in designing a 3D vision foundation model with a new representation and proposing a new perspective for end-to-end metric indoor scene reconstruction. Currently, we train with two-view inputs at \\$512 \\times 384\\$ resolution, which already imposes high GPU demands (Section 4.1). While we have not yet conducted end-to-end multi-view training, Plana3r supports multi-view reconstruction at inference time with multiple feed-forwards. Extending to full multi-view training is straightforward—e.g., by scaling the data and following approaches like VGGT [1]. We appreciate the suggestion and agree that one-pass multi-view reconstruction is a promising direction. We plan to explore this when we have sufficient computational resources in the future.\n\n### 4. Failure case analysis:\nAs shown in Figures 3, 4, 5, 7, and 8, our method will not fail in low-texture areas and can yield good plane segmentation and planar reconstruction. For the complex clutter scene, if there are many non-planar areas and very complex textures in the scene, such as potted plants, etc., it is difficult for our method to achieve high-quality reconstruction. However, the motivation of this paper is for the man-made, structured indoor scene.\n\n[1] VGGT: Visual Geometry Grounded Transformer (CVPR 2025)\n\n[2] Continuous 3D Perception Model with Persistent State (CVPR 2025)\n\nWe hope these clarifications address your concerns. Thank you again for your valuable feedback, which has strengthened our manuscript."}, {"review_id": "QJvWfludww", "reviewer": "Reviewer_3Lx4", "summary": "The paper presents a method for indoor metric 3D reconstruction. Given two input views, the method employs Vision Transformers to predict a set of planar primitives as well as the relative camera poses. It uses only depth and normal information for supervision, thereby bypassing the need for 3D annotations and enabling training on multiple large-scale stereo datasets.", "questions": "* In the planar reconstruction results, how are the Chamfer distance and F-score calculated using the reconstructed planes? Are points uniformly sampled from the planar surfaces for this evaluation?\n* For cases similar to the second row of Figure 7, what does the method predict for the relative camera pose when there is minimal or no overlap between the two input images?\n* How are image pairs selected from the datasets to ensure overlap? Or are the image pairs chosen randomly?", "limitations": "It would be beneficial to discuss whether the degree of view overlap poses a challenge for the proposed method.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n* The paper is well written and easy to follow.\n* The experiments cover many aspects and demonstrate promising results.\n\n\nConcerns:\n* Some evaluation details are not clearly explained in the paper. Specific questions are listed below.\n* Since the proposed method focuses on planar reconstruction and includes a plane merging step (Section 3.4), the comparison would be more informative if a simple RANSAC-based plane fitting algorithm were applied to the baselines that output point clouds.\n* In two-view reconstruction problems, one might assume there is some overlapping area between the views. It would be helpful to discuss how the overlap ratio affects performance. Given that the method is trained on indoor datasets, it would also be interesting to see if these datasets contain strong priors that allow the model to \"guess\" the relative pose and geometry even when the two views have little or no overlap.", "was_revised": true, "final_justification": "The paper presents a method for indoor metric 3D reconstruction using planar primitives and relative pose estimation from two views. The problem is well motivated, as indoor scenes consist of many planar structures, making point-wise representations potentially excessive and less stable. The proposed method is evaluated on large-scale datasets and demonstrates performance that outperforms state-of-the-art point-based methods. The authors have addressed the concerns raised in the reviews, including the influence of view overlap, and have also shown that the method generalizes to a multi-view setup. They have committed to further polishing the final version. Given these reasons, I maintain my initial recommendation for acceptance.", "rebuttal": "We sincerely thank you for your positive feedback on our manuscript, especially your recognition of our model’s strong performance and comprehensive experiments. We greatly appreciate your constructive comments and address your questions in detail below. \n\n### 1. On the Chamfer distance and F-score calculation\nYour understanding is correct, we uniformly sample points from the planar surfaces for this evaluation. \n\n### 2. Comparison with RANSAC-based plane fitting baseline\nWe supplement a new RANSAC-based plane fitting baseline here. Our method achieves much better performance.\n| Method         | ScanNetV2               |                       | Matterport3D            |                       |\n|----------------|-------------------------|-----------------------|--------------------------|-----------------------|\n|                | Chamfer ↓               | F-score ↑             | Chamfer ↓                | F-score ↑             |\n| Mast3R         | 0.21                    | 74.92                 | 0.49                     | 30.01                 |\n| Mast3R+RANSAC  | 0.20                    | 73.12                 | 0.49                     | 28.35                 |\n| Ours           | **0.11**                | **92.52**             | **0.32**                 | **56.63**             |\n\n### 3. How we sample pairs as training set\nYour understanding is correct—thank you for the insightful observation. In our training set (totally 4M image pairs),  we include approximately 0.57M image pairs with no overlap, while the remaining 3.43M pairs are randomly sampled from nearby frames (mainly within the next 10 frames). We observed that incorporating the 0.57M non-overlapping pairs helps the model learn strong geometric priors, enabling it to infer relative pose and scene geometry even when the two views have little or no overlap. Below, we provide a quantitative analysis to show the impact of including the 0.57M non-overlapping image pairs. \n\n### 4. Performance analysis on pair’s overlap\nWe conduct an additional ablation study to evaluate the impact of incorporating the 0.57M non-overlapping image pairs on model performance during training. In addition, we analyze how the overlap ratio affects performance. Specifically, we define the overlap ratio as the maximum percentage of pixels in one image (either the first or the second) that have direct correspondences in the other.\n\nAs shown in the table below, including these non-overlapping pairs significantly improves the model’s overall performance. Furthermore, we observe that as the overlap ratio in the test set decreases, the model’s accuracy consistently degrades.\n\n**ScannetV2** (wo/ExtData means training without 0.57M non-overlapping image pairs):\n| Overlap                |Training Data| Translation |  |   |   | Rotation|  |   |\n| ---------------------- |-------------| ------- | ------- | ------ | - | --------- | --------- | ----- |\n|                 || Med.(m)↓ | Mean(m)↓ | <=0.2m↑  |   | Med.(deg) ↓| Mean(deg)↓ | <=10°↑  |\n| Easy (>50%)            |Full| 0.05    | 0.07    | 97.40  |   | 1.53      | 1.80      | 99.72 |\n|                                 | wo/ExtData            | 0.05    | 0.09    | 95.73  |   | 1.77      | 2.77      | 99.16 |\n| Medium (15%-50%)     |Full  | 0.06    | 0.10    | 93.56  |   | 2.02      | 2.68      | 98.88 |\n| | wo/ExtData            | 0.07    | 0.15    | 89.68  |   | 2.25      | 4.24      | 97.60 |\n| Hard (<15%) |Full           | 0.12    | 0.26    | 71.98  |   | 2.68      | 5.53      | 95.67 |\n||wo/ExtData            | 0.15    | 0.41    | 63.03  |   | 3.46      | 10.35     | 88.30 |\n| Very Hard (non-overlap)| Full | 0.19    | 0.37    | 51.26  |   | 2.87      | 7.45      | 94.12 |\n| |wo/ExtData            | 0.23    | 0.53    | 46.22  |   | 4.16      | 16.64     | 79.83 |\n| All | Full                   | 0.07    | 0.13    | 89.16  |   | 2.01      | 3.16      | 98.30 |\n|     |wo/ExtData            | 0.07    | 0.20    | 84.60  |   | 2.31      | 5.38      | 95.68 |\n\n-----------------------------------------------------------------------------------------\n\n**MP3D**  (wo/ExtData means training without 0.57M non-overlapping image pairs):\n| Overlap                |Training Data| Translation |  |   |   | Rotation|  |   |\n| ---------------------- |-------------| ------- | ------- | ------ | - | --------- | --------- | ----- |\n|                 || Med.(m)↓ | Mean(m)↓ | <=0.2m↑  |   | Med.(deg) ↓| Mean(deg)↓ | <=10°↑  |\n| Easy (>50%)            | Full        | 0.20    | 0.33    | 49.72  |   | 1.65      | 2.37      | 99.15 |\n|                        | wo/ExtData  | 0.23    | 0.49    | 44.84  |   | 1.92      | 4.29      | 97.08 |\n| Medium (15%-50%)       | Full        | 0.25    | 0.45    | 37.55  |   | 2.19      | 4.46      | 96.41 |\n|                        | wo/ExtData            | 0.29    | 0.66    | 36.75  |   | 2.41      | 7.67      | 91.81 |\n| Hard (<15%)            | Full        | 0.46    | 0.82    | 17.35  |   | 2.96      | 11.12     | 85.97 |\n|                        | wo/ExtData            | 0.59    | 1.03    | 14.25  |   | 3.18      | 17.84     | 79.23 |\n| Very Hard (non-overlap) | Full        | 0.82    | 1.11    | 5.63   |   | 3.00      | 12.37     | 85.28 |\n|                        | wo/ExtData            | 0.85    | 1.11    | 5.19   |   | 2.98      | 18.84     | 79.65 |\n| All                    | Full                  | 0.24    | 0.45    | 40.16  |   | 2.00      | 4.49      | 96.12 |\n|                        | wo/ExtData            | 0.29    | 0.64    | 37.14  |   | 2.24      | 7.62      | 92.37 |\n\nWe hope these clarifications and supplement results effectively address your feedback. Thank you again for your valuable insights, which have significantly strengthened our manuscript."}]

Author rebuttal:

Meta-review: The paper introduces a pose-free framework for metric-scale planar 3D reconstruction from unposed two-view images, leveraging sparse planar primitives and supervision from depth and normal maps without requiring plane annotations. Reviewer’s feedback was largely positive, noting the paper’s strengths while requesting clarification on several points, including (1) whether the model can handle more than two views, (2) the computational cost, (3) missing evaluation details, and (4) the effect of view overlap. The rebuttal successfully addressed these concerns, convincing even the initially negative reviewer. Consequently, all reviewers recommended acceptance.
Decision: Accept (poster)