Title: PLANA3R: Zero-shot Metric Planar 3D Reconstruction via Feed-forward Planar Splatting
Venue: NeurIPS 2025
Accepted: yes
Keywords: 3D Reconstruction

Planar Reconstruction

Plane Segmentation
Abstract: This paper addresses metric 3D reconstruction of indoor scenes by exploiting their inherent geometric regularities with compact representations. Using planar 3D primitives -- a well-suited representation for man-made environments -- we introduce PLANA3R, a pose-free framework for metric $\underline{Plana}$r $\underline{3}$D $\underline{R}$econstruction from unposed two-view images. Our approach employs Vision Transformers to extract a set of sparse planar primitives, estimate relative camera poses, and supervise geometry learning via planar splatting, where gradients are propagated through high-resolution rendered depth and normal maps of primitives. Unlike prior feedforward methods that require 3D plane annotations during training, PLANA3R learns planar 3D structures without explicit plane supervision, enabling scalable training on large-scale stereo datasets using only depth and normal annotations. We validate PLANA3R on multiple indoor-scene datasets with metric supervision and demonstrate strong generalization to out-of-domain indoor environments across diverse tasks under metric evaluation protocols, including 3D surface reconstruction, depth estimation, and relative pose estimation. Furthermore, by formulating with planar 3D representation, our method emerges with the ability for accurate plane segmentation. The project page is available at: \url{https://lck666666.github.io/plana3r/}.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a method for indoor metric 3D reconstruction. Given two input views, the method employs Vision Transformers to predict a set of planar primitives as well as the relative camera poses. It uses only depth and normal information for supervision, thereby bypassing the need for 3D annotations and enabling training on multiple large-scale stereo datasets.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n* The paper is well written and easy to follow.\n* The experiments cover many aspects and demonstrate promising results.\n\n\nConcerns:\n* Some evaluation details are not clearly explained in the paper. Specific questions are listed below.\n* Since the proposed method focuses on planar reconstruction and includes a plane merging step (Section 3.4), the comparison would be more informative if a simple RANSAC-based plane fitting algorithm were applied to the baselines that output point clouds.\n* In two-view reconstruction problems, one might assume there is some overlapping area between the views. It would be helpful to discuss how the overlap ratio affects performance. Given that the method is trained on indoor datasets, it would also be interesting to see if these datasets contain strong priors that allow the model to \"guess\" the relative pose and geometry even when the two views have little or no overlap.\n\nQUESTIONS:\n* In the planar reconstruction results, how are the Chamfer distance and F-score calculated using the reconstructed planes? Are points uniformly sampled from the planar surfaces for this evaluation?\n* For cases similar to the second row of Figure 7, what does the method predict for the relative camera pose when there is minimal or no overlap between the two input images?\n* How are image pairs selected from the datasets to ensure overlap? Or are the image pairs chosen randomly?\n\nLIMITATIONS:\nIt would be beneficial to discuss whether the degree of view overlap poses a challenge for the proposed method.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-09"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes PLANA3R, a feed-forward, pose-free framework for metric-scale planar 3D reconstruction from unposed two-view images. It uses sparse planar primitives and learns from depth and normal maps without plane annotations and shows strong zero-shot generalization across indoor datasets.\n\nSTRENGTHS AND WEAKNESSES:\nStrengthens：\n- The use of planar primitives offers a concise and interpretable alternative to dense point clouds.\n- Extensive experiments demonstrate state-of-the-art performance across different tasks.\n- The approach trains without explicit plane-level supervision, relying only on depth and normal maps, which are more readily available.\n- PLANA3R predicts relative pose and 3D structure in a single forward pass, making it suitable for real-time applications.\n\nWeaknesses：\n- The method feels like a relatively direct combination of ideas from PlanarSplatting and DUSt3R/MASt3R, with limited conceptual novelty.\n- It is unclear how well it generalizes to multi-view or more complex scenes beyond pairwise settings.\n- The hierarchical prediction of 16× and 8× downsampled features constitutes a core design intended to select appropriate fine-grained textures in different regions to reduce computational overhead and improve inference speed. However, the authors have not yet provided quantitative evidence, such as measurements of runtime, memory consumption, or floating-point operations (FLOPs). Moreover, although the design is intended to accelerate computation, it requires computing gradient-based texture masks, which itself incurs additional computational cost. It remains unclear whether this overhead offsets the claimed improvement in computational speed.\n\nQUESTIONS:\n1. Can the model be extended to handle more than two input images? Have you experimented with applying the framework to reconstructions using a different number of input views?\n2. Could the authors provide a quantitative analysis of the computational performance achieved by the hierarchical feature design, including measurements of runtime, memory consumption, or floating point operations (FLOPs)?\n3. How much resource consumption and calculation speed increase does the design of image texture mask bring?\n4. Are there typical patterns or scenarios where the hierarchical design fails, e.g., in low-texture areas or complex clutter?\n\nLIMITATIONS:\nThe authors have adequately addressed the limitations and societal impact in the main paper.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-22"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes PLANA3R, a strong method that extends the 3R framework to handle structured indoor scenes. The key idea is to pretrain a feed-forward plane prediction network that learns to generate sparse planar primitives in metric scale, using only depth and normal maps as supervision from large indoor datasets. The differentiable PlanarSplatting module plays a key role for end-to-end training of the pipeline. PLANA3R predicts planar 3D primitives directly from two unposed images, supporting multiple downstream tasks such as depth estimation, plane segmentation, and reconstruction. Extensive experiments demonstrate that PLANA3R achieves impressive performance, showing clear improvements over prior methods in both qualitative visualizations and quantitative metrics.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n1. This paper is an interesting and meaningful extension of the 3R framework towards structured planar 3D reconstruction. By leveraging PlanarSplatting as a differentiable renderer, the entire training pipeline is made fully end-to-end trainable without explicit plane-level annotations, which is a notable technical strength.\n2. The proposed hierarchical primitive prediction combines low-resolution and high-resolution primitives to achieve a balance between primitive numbers and reconstruction fidelity, which is novel and practically effective.\n3. The experimental setup is thorough, covering diverse datasets and multiple tasks, with strong improvements over baselines in both quantitative metrics and visual quality.\n4. The paper is well-written, the technical presentation is clear, and the results are convincingly demonstrated.\n\nWeaknesses\n\nI did not find any major weaknesses. One minor point: I suggest the author to reorganize Figure 2 to more clearly illustrate how the low-resolution and high-resolution primitives are generated. As currently drawn, it may leave readers with the impression that the high-res primitives are simply upsampled directly from the low-res ones, which is not the case.\n\nQUESTIONS:\n1.\tThere are many incomplete depths in GT datasets because they are obtained from sensored LiDAR cameras. How do you deal with the incompleteness in GT depths?\n2.\tPLANA3R uses estimated normal from Metric3Dv2 in scenarios where there is no GT normal. However, we can also generate pseudo normal from GT depths just as what 2DGS does. It may provide more accurate normal supervision. Does the author observe any difference in quality or training stability between these two pseudo normals?\n3.\tWhy are the input images in Figure 3,4,5 with such low quality? Aren’t they come from GT image sequences?\n\nLIMITATIONS:\nNo evident limitations.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes PLANA3R, a two-view metric planar reconstruction method designed for indoor environments. Built upon the DUSt3R network framework, the authors introduce network heads that output multi-level planar primitives and utilize normal gradients to guide the selection of primitives, thereby reducing the number of primitives. Additionally, the method leverages normal and depth images from the dataset to supervise training, without requiring ground-truth plane annotations. Experimental results demonstrate that PLANA3R outperforms the baselines on tasks such as 3D surface reconstruction, depth estimation, and relative pose estimation.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n1. The motivation of the paper is clear: planar primitives are well-suited for representing lightweight, structured indoor environments. The Introduction and Related Work sections are logically organized and easy to follow. \n2. The use of multi-level, multi-resolution primitive outputs combined with a normal gradient-based selection mechanism effectively reduces the number of primitives.\n3. Experimental results demonstrate that the proposed method delivers strong performance across multiple tasks.\n\n**Weaknesses:**\n1. Since PlanarSplatting is a fundamental component of this work, it would be helpful to include a more detailed explanation of its core ideas to make the paper more self-contained and accessible to readers unfamiliar with it.\n2. The Method section lacks clarity in several important aspects, making it difficult to follow. For example, in Equation (1), the meaning of the indices $i$ and $j$ in $d_{\\pi}^{i,j}$, $\\mathbf{r}{\\pi}^{i,j}$, and $\\mathbf{q}{\\pi}^{i,j}$ is not clearly explained. Does $\\mathbf{q}{\\pi}^{i,j}$ represent the quaternion of a primitive generated from image $i$, expressed in the coordinate frame of camera $j$? or $\\mathbf{q}{\\pi}^{i,j}$ is simply under its camera coordinate system.  The lack of clarity regarding the coordinate systems involved makes it difficult to fully understand how the training losses are formulated and applied. \n3. In the *Training losses and training strategies* section, \"As a result, directly applying the differentiable planar primitive rendering .......\"\nActually, I don't understand what the rendering is used for. I couldn't find any information that explained it. Do you mean render the depth and normal images from image $i$ to camera $j$, or render images from all the primitives under the same coordinate system (use $q$)? And then use the rendered image to compare it with the ground truth? What's the target for introducing this rendering loss to the training stage? \n4. It is also not clear whether the warm-up training phase includes the pose loss. \n\nOverall, I appreciate the value of this work and its strong performance across different tasks. However, the lack of clarity in important implementation details makes it difficult to fully understand how the proposed method is realized. If the authors can provide more detailed explanations regarding these aspects, I would be willing to consider increasing my score.\n\nQUESTIONS:\nsee Weaknesses.\n\nLIMITATIONS:\nYes, in the supplementary.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}