Title: 3D Gaussian Flats: Hybrid 2D/3D Photometric Scene Reconstruction
Venue: NeurIPS 2025
Accepted: yes
Keywords: gaussian splatting

novel view synthesis

planar reconstruction
Abstract: Recent advances in radiance fields and novel view synthesis enable creation of realistic digital twins from photographs. However, current methods struggle with flat, texture-less surfaces, creating uneven and semi-transparent reconstructions, due to an ill-conditioned photometric reconstruction objective. Surface reconstruction methods solve this issue but sacrifice visual quality. We propose a novel hybrid 2D/3D representation that jointly optimizes constrained planar (2D) Gaussians for modeling flat surfaces and freeform (3D) Gaussians for the rest of the scene. Our end-to-end approach dynamically detects and refines planar regions, improving both visual fidelity and geometric accuracy. It achieves state-of-the-art depth estimation on ScanNet++ and ScanNetv2, and excels at mesh extraction without overfitting to a specific camera model, showing its effectiveness in producing high-quality reconstruction of indoor scenes.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a novel method that combines 3D Gaussians and 2D Gaussians binded on the detected planar to represent non-planar and planar areas, respectively. To optimize the hybrid representation, the authors propose a plane initialization method and multi-stage optimization process, which demonstrates helpful in the ablation study. The experiment results indicate the effectiveness of this paper to some extent.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:** \n\n1. This paper propose a hybrid representation to leverage the geometric prior of detected planars in the indoor scenes, which improves the quality of depth map estimation. \n2. The experiment results demonstrate this hybrid representation surpasses previous planar reconstruction methods on diverse datasets.\n\n**Weakness:**\n\n1. The major weakness of this paper is the confused application areas described in the introduction and experiment settings. For novel view synthesis, this method shows performance drop compared with 3DGS or 3DGS-MCMC. For surface reconstruction, this method can only reconstruct planar areas, with large missing regions for other areas (as shown in Fig. 6). Thus I concern whether comparing this method with 3DGS(NVS) or PGSR(Surface Rec) is necessary and fair. Fig 4 and Fig 5 only demonstrate this method can obtain better depth estimation results.\n2. For depth estimation, the comparison with 2DGS is also not very fair. The hybrid representation leverages planar prior to refine the depth map of planar regions, while 2DGS relys on no planar consumption and can generalize to ourdoor scenes. The comparison with 2DGS + planar regularizer (such as structnerf) is necessary to demonstrate the effectiveness of the hybrid representation, although the authors say \"regularizers can be difficult to tune\", which is not a straightforward conclusion to me.\n3. Some statements about the relationship between 2D Gaussians, 3D Gaussians and planar are confused in Sec 3. Is each 2D Gaussians binded extractly on the plane for its definition in Eq.(1) and (2)? If so, what's the smallest Euclidean distance of 2D Gaussians in line 182? Should it be always zero? If Sec 3.4 describes how to transform a 3D gaussian near the plane to a 2D Gaussian binded on the plane, I highly recommend do not use \"freeform\" or \"planar\" and change the subsection name, cause desifincation has specific meanings in 3D GS.\n4. In line 147, may M correspond to several planar masks? Because the author say subsequent RANSAC runs after detecting one valid plane.\n5. For experiments, in line 246, the authors mention this method generalize to different camera models well, while previous fails. The reason is unclear. This method relys heavily on the performance of PlaneRecNet. Is this robustness caused by the diverse training data of PlaneRecNet, which contains different types of camera models? Why previous methods fail?\n\nQUESTIONS:\nConcerns: See Weakness part.\n\nTypos:\n\n1. line 131, wrong ref for section 3.2\n2. line 140, no $d_{th}$\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents 3D Gaussian Flats, a hybrid novel view synthesis approach that proposes using both 3D gaussians and planar-constrained 2D gaussians. This allows it to come close to 3D Gaussian Splatting's rendering quality while improving upon its depth reconstruction in flat areas (ie: avoiding \"holes\" in apparent surfaces) via planar constraints. The method requires image masks for each plane instance (obtained via SAMv2) in addition to camera images/poses as in standard 3DGS. The optimization begins with standard 3D gaussian optimization as in 3DGS before initializing planar surfaces. The planar surfaces are constructed by projecting 3D gaussians onto the image masks, and then running RANSAC based on the gaussian means. 3D Gaussians within a certain range to the plane are then relocated to planar-aligned 2D gaussians. Optimization then alternates between gaussian optimization and the optimization of planar parameters.\n\nThe approach is evalauted on indoor scenes from ScanNet++ and ScanNetv2 against common baselines such as 3DGS, MCMC, and 2DGS. It outperforms all baselines with respect to depth metrics, and is competitve with regards to appearance metrics such as PSNR. Mesh extraction is also evaluated, and the approach outperforms the Airplanes and PlanarRecon baselines.\n\nSTRENGTHS AND WEAKNESSES:\nOverall the paper writing is clear (although the figures could be enlarged/improved - especially the teaser where I don't think it's clear what \"faux render\" means) and the hybrid approach seems sensible. The results on ScanNet look compelling and the ablations are reasonable.\n\nThe main weakness I see is its reliance on accurate planar segmentation masks and its potential brittleness when the masks are not totally accurate. Furthermore, the evaluation section limits itself to ScanNet scenes whereas a wider evaluation would be interesting (Fig 1 shows a MipNeRF 360 scene but MipNeRF is not evaluated quantitatively). Even if dense ground truth depth is hard to find for most datasets, getting PSNR metrics on M360 for example would give us a sense of how much the hybrid approach degrades visual quality. And even if outdoor scene reconstruction is not a focus of this paper, it would be interesting to evaluate things like road reconstruction on AV datasets like Waymo or KITTI (which also contain sparse depth that could be used for evaluation), or on Tanks and Temples which comes with (slightly imperfect) laser-scan derived meshes.\n\nQUESTIONS:\nI provisionally feel positive about the paper. The main thing I'd be interested in seeing are even qualitative results on outdoor scenes such as in AV to see if road reconstruction looks better than in 3DGS and/or Tanks and Temples.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-25"}

comments: SUMMARY:
3D Gaussian Flats proposes a hybrid 2D/3D representation that jointly optimizes constrained planar (2D) Gaussians for modeling flat surfaces and freeform (3D) Gaussians for the rest of the scene. This end-to-end approach dynamically detects and refines planar regions within the scene. The core innovation lies in this adaptive, dual-representation strategy, enabling the creation of photorealistic digital twins that are robust to texture-less regions. The performance is supported by extensive experiments on ScanNetV2 and ablation studies.

STRENGTHS AND WEAKNESSES:
**Strengths**
1. It is interesting to segment planar objects from the scene and condition separate Gaussians to model them. This hybrid 2D/3D representation efficiently improve the reconstruction and view synthesis quality.
3. The paper is well-motivated and the structure is clean and easy to understand.

**Weaknesses** (including questions)
1. How does the method apply to outdoor scenes that also have planar parts? The author should include more experiments rather than only focusing on indoor scenes.
2. The qualitative comparison in Figure 1, 4, and 7 demonstrates marginal improvement in terms of visual quality. In addition, the figure seems quite low-res., potentially directly from sceenshots.
3. It is quite confusing that in Figure 6, 3D Gaussian Flats outperforms Airplanes in terms of “Acc” but inferior in “Comp.” And “Chamfer Distance”. The author should provide more explaination regarding this.

QUESTIONS:
See "Strengths And Weaknesses" section.

LIMITATIONS:
See "Strengths And Weaknesses" section.
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 2
DATE: 2025-06-29

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a novel method for representing 3D scenes by combining 2D and 3D Gaussians. Specifically, 2D Gaussians are employed for planar surfaces, while 3D Gaussians handle the remaining scene elements. The selection of 2D Gaussians is based on their projection onto pre-estimated plane masks and their adherence to a RANSAC-based plane fitting, allowing the method to automatically differentiate between 2D and 3D Gaussian assignments.\n\nSTRENGTHS AND WEAKNESSES:\n--- Strengths ---\n\nThe concept of integrating both 2D and 3D Gaussians within a single scene representation is interesting. However, the current experimental setup does not sufficiently motivate or justify the advantages of this hybrid approach.\n\n--- Weaknesses ---\n\nThe paper in its current state shows several major weaknesses. \n\n- Limited Justification for Hybrid Approach: The proposed method for identifying 2D Gaussians - relying on projections into pre-estimated plane masks and RANSAC for robustness - lacks a clear advantage over existing techniques. It is unclear how this approach surpasses methods that utilize monocular depth or normal supervision, such as DNSplatter, in terms of geometric accuracy. Furthermore, given that 2DGS has already demonstrated superior geometric accuracy with disc-based representations compared to 3D Gaussians, the rationale for a hybrid approach rather than exclusively using 2D discs is not well-motivated.\n- Insufficient Experimental Validation: The experimental section is very weak, particularly concerning geometric accuracy comparisons. The paper primarily focuses on photometric/depth accuracy and some geometric comparisons with planar segmentation methods, neglecting crucial baselines. Essential comparisons with state-of-the-art 3D Gaussian Splatting (3DGS) methods that prioritize geometric accuracy (e.g., F1 score of the extracted mesh), such as DNSplatter, Gaussian Opacity Fields, and 2DGS, are missing. A table presenting geometric accuracy metrics (mesh extraction quality) across standard datasets like Tanks and Temples, DTU, or ETH3D would significantly strengthen the paper.\n- Incomplete Mesh Extraction Evaluation: The current mesh extraction evaluation is limited to planar surfaces. To provide a comprehensive assessment of geometric accuracy for the entire scene, the authors should apply and evaluate a robust mesh extraction method, potentially leveraging the approach demonstrated in 2DGS, which appears straightforward to implement.\n\nQUESTIONS:\n- What is the mesh quality compared to 2DGS, Gaussian Opacity Fields and DNSplatter?\n- How is the method different than simply having monodepth/mononormal supervision? \n- What RANSAC did the authors use to get the plane parameters? \n- What is the advantage of using a mix of 2D/3D Gaussians instead of only 2D ones? \n\nI am willing to improve my score, but only if the authors show mesh quality comparisons on standard datasets against 2DGS, Gaussian Opacity Fields and DNSplatter.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-01"}