Title: GeGS-PCR: Fast and Robust Color 3D Point Cloud Registration with Two-Stage Geometric-3DGS Fusion
Venue: NeurIPS 2025
Accepted: yes
Keywords: Point Cloud Registration; Geometric-Color Fusion; Geometric-3DGS; Color Encoding; LORA; Differentiable Rendering; Photometric Loss; Low Overlap Scenarios; Robust Registration; Multi-Stage; 3D Point Cloud Processing;
Abstract: We address the challenge of point cloud registration using color information, where traditional methods relying solely on geometric features often struggle in low-overlap and incomplete scenarios. To overcome these limitations, we propose GeGS-PCR, a novel two-stage method that combines geometric, color, and Gaussian information for robust registration. Our approach incorporates a dedicated color encoder that enhances color features by extracting multi-level geometric and color data from the original point cloud. We introduce the Geometric-3DGS module, which encodes the local neighborhood information of colored superpoints to ensure a globally invariant geometric-color context. Leveraging LORA optimization, we maintain high performance while preserving the expressiveness of 3DGS. Additionally, fast differentiable rendering is utilized to refine the registration process,  leading to improved convergence. To further enhance performance, we propose a joint photometric loss that exploits both geometric and color features. This enables strong performance in challenging conditions with extremely low point cloud overlap. We validate our method by colorizing the Kitti dataset as ColorKitti and testing on both Color3DMatch and Color3DLoMatch datasets. Our method achieves state-of-the-art performance with Registration Recall at 99.9%, Relative Rotation Error as low as 0.013, and Relative Translation Error as low as 0.024, improving precision by at least a factor of 2.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis manuscript proposes a novel colored-point-cloud registration method by incorporating color, coordinates and 3DGS together. Specifically, the method is built upon ColorPCR with three key improvements:\n\n1.\tAdding an MLP for better color feature extraction.\n\n2.\tAdding Gaussian representation of superpoint neighborhoods, which enhances feature extraction, position embedding during GeoTransformer-type superpoint attention, and differentiable rendering for pose refinement.\n\n3.\tAdding a differentiable rendering loss (i.e., photometric loss) for pose refinement.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1.\tThe paper is well structured and written. it is easy to follow the authors’ ideas. The idea is original and moderately significant, where applying 3DGS on superpoint-matching is novel and possibly took inspiration from GMM-based registration methods. The paper also incorporated a theoretical proof for convergence guarantees of the 2nd stage pose refinement.\n\n2.\tThe core idea of applying gaussian during superpoint matching makes a lot of sense to me. Superpoint represents a small piece of local geometry which is highly likely anisotropic. Such property can be well captured with Mahalanobis distance instead of Euclidean distance in favor of superpoint matching and alignment.\n\n3.\tDesign spaces are well explored, although I feel sorry about the addition of many tunable parameters.\n\nWeaknesses:\n\n1.\tWhile performance increments are consistent, the tested benchmarks are quite saturated, and the absolute improvements are not significant.\n\n2.\tAblation experiments are not well conducted, which IMHO is very important to validating the effectiveness for all your added components. See Q2 for details.\n\n3.\tMany niche designs have lots of parameters, but lack corresponding ablation experiments. See Q3 for details.\n\nAdvice (not a strength or weakness):\n\n1.\tIt is viable to use KITTI-360 for coloring surround-view KITTI point clouds, although this would create an alternative benchmark and requires re-running the baselines. I would also recommend using other autonomous-driving datasets such as nuScenes or Waymo with surround-view color images.\n\nQUESTIONS:\n1.\tDesign of Eq. 5 is questionable. The concatenated components have different meanings along different dimensions, but are compressed together with a single top-k operation. That way the model will not know the meaning of a specific dimension after the top-k, highly possibly causing confusion.\n\n2.\tAblation is under-optimized and needs explanation. Here I assume that your work is completely based on Color-PCR (which I think is self-evident, but please point out if otherwise). While the ablation only provides deduction of single components from GeGS-PCR, from retrospective we can also see them as incomplete increments from ColorPCR to GeGS-PCR. However, all the ablated models perform much worse than ColorPCR, comparing ablation figures in Table 3 and Table 8 against ColorPCR figures in Tab. 2. For example, does Tab. 3 line (a) hint that ColorPCR + Color Decoder + 3DGS + Geometric PE deteriorates registration performance compared to barebone ColorPCR?\tMaybe you can approach this concern by adding a fair baseline in Tab. 3 and describing the ablation components more clearly.\n\n3.\tMany designs are not ablated. E.g., does the design of Eq. 3 achieve the optimal color encoding performance, compared to using no alpha, or applying alpha on all layers? What is the effect of tuning lambda in Eq. 4, k in Eq. 5, lambda in L213 (duplicate notation), gamma in Eq. 6, gamma in Eq. 10 (duplicate notation), lambda in Eq. 11(2nd duplicate notation), \\sigma_{GS} in Eq. 19 and lambda in Eq. 21(3rd duplicate notation)? The list continues. Maybe you can approach this concern by both reducing the number of manually-tuned parameters and doing ablation when necessary. You can simply add some descriptions on feasible value ranges from your experience for less important parameters. However, leaving all those parameters with no value specification is not acceptable.\n\nOverall, the paper presents a novel idea that seems to work perfectly consistent during experiments. However, the severely under-qualified and inconsistent ablation greatly undermines the credibility of such experimental results, causing me to rate a borderline reject. I would be happy to raise my ratings if my major concerns are addressed.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces GeGS-PCR, a novel two-stage method for 3D point cloud registration that integrates geometric, color, and Gaussian (3DGS) information to achieve robust performance, particularly in low-overlap and incomplete scenarios. The method employs a dedicated color encoder to extract noise-robust color features, a Geometric-3DGS module to encode local neighborhood information of superpoints, and a joint photometric loss with differentiable rendering for refined registration. The authors also create a new dataset, ColorKitti, by colorizing the Kitti dataset to validate the method's generalization. Experimental results on Color3DMatch, Color3DLoMatch, and ColorKitti demonstrate state-of-the-art performance.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: \n\n1. The paper presents a high-quality contribution with rigorous experimental validation. The proposed GeGS-PCR method achieves improvements over state-of-the-art methods (e.g., ColorPCR, GeoTransformer) across multiple metrics (FMR, IR, RR, RRE, RTE) on Color3DMatch and Color3DLoMatch datasets.  The creation of ColorKitti enhances the evaluation scope, addressing the scarcity of colored point cloud datasets. The reproducibility is well-supported with open-access code, datasets, and detailed experimental settings.\n\n2. The paper is well-structured, with clear descriptions of the problem, method, and experimental setup. The pipeline (Fig. 2) effectively illustrates the coarse-to-fine approach, and mathematical formulations (e.g., covariance matrix, photometric loss) are precise and well-explained. The use of figures (e.g., Fig. 3, 7, 8) and tables (e.g., Tables 1, 2, 7, 8) enhances understanding of qualitative and quantitative results. \n\nWeaknesses: \n\n1. The performance improvement over ColorPCR, while notable, is marginal in some metrics (e.g., 0.4% increase in RR on C3DM, 1.3% in IR), raising questions about the practical significance of the added complexity. The computational complexity of the Geometric-3DGS module, particularly in high-overlap scenarios, is acknowledged but not quantified. The paper lacks a detailed comparison of computational resources (e.g., GPU memory usage, training time) against baselines, which is critical for assessing practical deployability. Additionally, the reliance on superpoint downsampling may limit performance in scenarios with irregular point density, which is not explored. The ablation study in Table 3 is not sufficiently clear, making it difficult to discern the precise impact of each component (e.g., color encoder vs. 3DGS) due to overlapping metric improvements and lack of detailed analysis.\n\n2. The significance is somewhat limited by the focus on specific datasets (Color3DMatch, Color3DLoMatch, ColorKitti), with no evaluation on other real-world datasets (e.g., ScanNet, Waymo). The marginal performance gains over ColorPCR may reduce the perceived impact, especially given the increased computational complexity.\n\nQUESTIONS:\nSee weakness.\n\nLIMITATIONS:\nThe authors adequately address the limitations and potential negative societal impacts. They acknowledge that GeGS-PCR’s reliance on superpoint downsampling leads to high memory and computational overhead in high-overlap scenarios and note the limitation of focusing on superpoint-level rather than scene-level registration. These are discussed in Section A.6, with future directions proposed (e.g., leveraging semantic scene understanding). The societal impact section highlights positive applications (e.g., autonomous navigation, urban planning) and negative risks (e.g., privacy concerns in surveillance, data manipulation), with mitigation strategies like gated releases suggested. The transparency in addressing these points is commendable and aligns with ethical research practices.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-16"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the challenge of robust 3D point cloud registration, particularly in scenarios with low overlap and incomplete geometry, by leveraging both geometric and color information. The authors propose GeGS-PCR, a novel two-stage method integrating a dedicated color encoder and a Geometric-3DGS (3D Gaussian Splatting) module. The approach tightly fuses geometric and color data via a dual-modal encoder and models the local neighborhood with 3D Gaussian parameters, further optimized by LoRA to reduce computational cost. A differentiable rendering and a joint photometric loss are introduced in the fine registration stage for further refinement. The method is validated on both indoor (Color3DMatch, Color3DLoMatch) and outdoor (ColorKitti) colored point cloud datasets, demonstrating state-of-the-art performance, especially under low-overlap conditions. Ablation studies and qualitative results are presented to justify each module's effectiveness.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\nTechnical Novelty & Originality:\nThe combination of geometric and color features at a deep level, the introduction of the Geometric-3DGS module, and the use of differentiable rendering for point cloud registration are all novel. The integration of LoRA for efficient parameterization is innovative and relevant for scalability.\n\nQuality & Thoroughness:\nThe methodology is clearly described, including mathematical formulation, network architecture, and optimization strategies. Both coarse and fine registration pipelines are well justified.\n\nEmpirical Evaluation:\nExtensive experiments on multiple benchmarks (Color3DMatch, Color3DLoMatch, ColorKitti) convincingly demonstrate the method's superiority, especially in low-overlap and noisy scenarios. Ablation studies support the necessity of each component.\n\nClarity:\nThe paper is generally well-organized. Key ideas, architecture, and modules are well-illustrated and justified with both qualitative and quantitative results.\n\nReproducibility:\nImplementation details, hyperparameters, dataset splits, and a public code repository are provided. Training/test procedures are described clearly.\n\nWeaknesses:\n\nScalability / Computational Cost:\nThe paper acknowledges that using superpoints and 3DGS parameterization may incur significant memory and computation overhead in high-overlap or very large scenes, but lacks a deeper discussion or experiments on scalability to truly large-scale datasets or real-time constraints.\n\nLimited Statistical Analysis:\nWhile results are comprehensive, the paper does not provide error bars or statistical significance analysis, making it harder to assess the robustness of the improvements.\n\nAblation on Individual Modalities:\nAlthough ablation is done, there is limited exploration into failure cases where color might hurt registration (e.g., severe color noise, lighting changes, or sensor mismatch), or where geometric features are insufficient.\n\nPotential Overfitting to Synthetic Colorization:\nColorKitti is constructed by colorizing KITTI point clouds via projection, which may not reflect real-world sensor noise or missing color. The generalizability to native colored LiDAR or different sensors is not verified.\n\nBroader Impacts & Safeguards:\nThe broader societal impact is discussed, but concrete safeguards or risk mitigation for data/model misuse are not detailed.\n\nQUESTIONS:\nScalability Experiments:\nHave the authors evaluated the memory and computation bottlenecks of GeGS-PCR on very large-scale or real-time datasets (e.g., full-scale outdoor driving datasets with hundreds of millions of points)? What strategies could further mitigate memory usage?\n\nRobustness to Color Noise/Distortion:\nHow does the method perform if the color channels are corrupted (e.g., illumination changes, sensor noise, partial missing color)? Would incorporating color confidence weights improve robustness?\n\nGeneralization Beyond Synthetic Colorization:\nSince ColorKitti is constructed by projecting color onto LiDAR points, have the authors tested the method on naturally colored point cloud datasets (e.g., from RGB-D sensors or mobile mapping platforms)? How does the method generalize to such data?\n\nFailure Modes:\nAre there scenarios where color information may harm registration, for example, in the presence of repetitive textures or objects with similar color but different geometry? How does GeGS-PCR handle such cases?\n\nStatistical Significance:\nCan the authors provide statistical analysis (e.g., multiple runs, error bars) to demonstrate the significance and robustness of their improvements?\n\nLIMITATIONS:\nThe authors have adequately discussed the limitations, especially regarding memory usage and computation overhead in high-overlap scenarios, as well as the dependency on superpoint extraction. They also note the need for scene-level registration and semantic integration in future work.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces GeGS-PCR, a two-stage pipeline for pairwise 3D point-cloud registration under low overlap and incomplete geometry. Stage 1 performs coarse alignment with a Color-Enhanced Geometric-3D Gaussian-Splatting (3DGS) representation: a learned color encoder is fused with down-sampled super-points that are parameterised as anisotropic Gaussians; a dedicated 3DGS self-attention transformer yields an overlap-invariant global feature. Stage 2 refines the pose by jointly minimising a differentiable photometric-plus-geometric loss, rendered from the fused 3DGS scene. Efficiency is maintained with LoRA compression of attention weights. Extensive experiments on Color3DMatch / Color3DLoMatch and a new outdoor ColorKITTI benchmark show state-of-the-art registration recall and markedly lower RTE/RRE compared to recent baselines. The appendix provides reproducibility materials, ablations, theoretical convergence proofs, and a discussion of memory overhead and scene-level extensions.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n1. Comprehensive evaluation across three benchmarks, large ablation suite (removal of color, 3DGS, photometric loss).  Well-designed loss with proof of Lipschitz convergence.\n2. Clear problem set-up and modular description; appendix lists hyper-parameters and scripts. \n3. Brings 3D Gaussian Splatting, hot in 3D rendering, to registration ; delivers new SOTA on low-overlap indoor/outdoor data; public ColorKITTI is likely to spur follow-up work.\n4. Novel combination of colour encoding + 3DGS + joint photometric optimisation; first to apply 3DGS attention to registration.\n\n**Weaknesses**\n\n1. Runtime/memory not fully quantified: only qualitative note that 3DGS adds “some” overhead; no FPS numbers on KITTI. Coarse-to-fine pipeline cannot yet perform scene-level global registration, acknowledged by authors. \n2. Main text occasionally defers key algorithmic details (e.g., 3DGS attention formulation) to appendix, hampering standalone readability.\n3. Impact may be limited for lidar-only pipelines without colour.\n\n4. Method builds on several recent ideas (Gaussian-Splatting, transformer attention, LoRA) in an incremental fashion; theoretical novelty is moderate.\n\nQUESTIONS:\n1. Please report end-to-end latency (ms) and peak VRAM for both stages on KITTI and 3DMatch, and compare against GeoTransformer/ColorPCR. This will clarify practical deployability.\n2. Have you tested performance when colour channels are synthetically perturbed (e.g., random hue shift) or in night-time lidar scans? An ablation would show whether the geometric branch alone suffices.\n3. Table 3 shows faster convergence, but how much accuracy is lost if LoRA is removed? Reporting final metrics with/without LoRA would justify this choice.\n\nLIMITATIONS:\n1. Memory & Latency Overhead. The 3D Gaussian-Splatting representation adds mean/covariance/opacity parameters, pushing GPU usage to ~11 GB in dense indoor scenes; no FPS or memory numbers are reported for KITTI, so deployment on edge devices remains uncertain.\n2. Strong Reliance on RGB. Both stages leverage colour cues; performance under poor illumination, night-time, or colour-missing LiDAR‐only data is untested, leaving robustness to colour noise unclear.\n3. Pairwise-only Rigid Alignment. The pipeline estimates a single SE(3) transform per pair; multi-view or scene-level registration (e.g., full SLAM) is not evaluated, limiting direct applicability to sequential mapping tasks.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}