Title: Reasoning Beyond Points: A Visual Introspective Approach for Few-Shot 3D Segmentation
Venue: NeurIPS 2025
Accepted: yes
Keywords: Point Cloud Semantic Segmentation

Few-shot Learning

Visual Introspective Prototype
Abstract: Point Cloud Few-Shot Semantic Segmentation (PC-FSS) aims to segment unknown categories in query samples using only a small number of annotated support samples. However, scene complexity and insufficient representation of local geometric structures pose significant challenges to PC-FSS. To address these issues, we propose a novel pre-training-free Visual Introspective Prototype Segmentation network (VIP-Seg). Specifically, we design a Visual Introspective Prototype (VIP) module that employs a multi-step reasoning approach to tackle intra-class diversity and domain gaps between support and query sets. The VIP module consists of a Prototype Enhancement Module (PEM) and a Prototype Difference Module (PDM), which work alternately to progressively refine prototypes. The PEM enhances prototype discriminability and reduces intra-class diversity, while the PDM learns common representations from the differences between query and support features, effectively eliminating semantic inconsistencies caused by domain gaps. To further reduce intra-class diversity and enhance point discriminative ability, we propose a Dynamic Power Convolution (DyPowerConv) that leverages learnable power functions to effectively capture local geometric structures and detailed features of point clouds. Extensive experiments on S3DIS and ScanNet demonstrate that our proposed VIP-Seg significantly outperforms current state-of-the-art methods, proving its effectiveness in PC-FSS tasks. Our code will be available at https://github.com/changshuowang/VIP-Seg .


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper mainly addresses the two challenges in the point cloud few-shot semantic segmentation, including feature misalignment between different domains and semantic class representation differences. In feature extraction, DyTaylorConv captures detained point features and improves intra-semantic difference. In addition, the main contribution lies in adopting the idea of Chain-of-Thought and using Visual Introspective Prototype (VIP), consisting of iterative PEM and PDM, for addressing the main challenges. Experiments show the SOTA performance comparing the current works on indoor datasets, and ablation studies show the effectiveness of VIP for few-shot learning in semantic segmentation.\n\nSTRENGTHS AND WEAKNESSES:\nstrengths:\n\n- The idea of using chain-of-thought in the 3D vision task is interesting, and the ablation studies show its significant improvement on the semantic segmentation performance. \n- Sufficient ablation studies are performed. Most hyperparameters are studied through experiments.\n\nweaknesses:\n\n- Contribution Confusion: The authors include DyTaylorConv in the contributions. However, the ablation studies show DyTaylorConv contributing little to the final performance, and the logical relationship between DyTaylorConv and the core contribution of the paper, Visual Introspective Prototype, is confusing.\n- No Inferencing Efficiency Evaluation: Though the training efficiency is shown, the iterative VIP structure may take more time in the inference compared to the other methods. But no inferencing efficiency evaluation is provided.\n- Few visualizations: Few visualizations are provided in this paper, making it unclear how complex indoor scenes the proposed method is applied to.\n\nQUESTIONS:\n- Further explain the relationship between DyTaylorConv and the Visual Introspective Prototype. A question should be answered: why can the two contributions be put in one paper?\n- Perform inferencing efficiency evaluation to show the efficiency of applying the iterative VIP.\n- Provide the visualization of the segmentation results. \n- I suggest further explanation of the experiment results. For example, explain why improvement is best on the 3-way-5-shot, which gives the most examples for few-shot learning, addressing the main challenges through the proposed method. This will further show the significance of the contributions.\n\n- For FEM and FDM, the current statements are still little confusing about how the structure designs improve the discriminability of prototype features and reduce intra-class diversity. I suggest directly explaining the function of each design for the challenges to make the contributions clearer.\n\nLIMITATIONS:\nYes. The authors clearly claim the limitation on single modality and single field application.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-21"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces VIP-Seg, a pre-training-free framework for Point Cloud Few-Shot Semantic Segmentation (PC-FSS), incorporating two key modules: Dynamic Taylor Convolution (DyTaylorConv) and a Visual Introspective Prototype (VIP) reasoning mechanism composed of PEM and PDM submodules. The proposed method aims to address intra-class diversity and domain gaps, achieving promising results on S3DIS and ScanNet.\n\nSTRENGTHS AND WEAKNESSES:\n1. Practical Motivation: Removing the need for pre-training improves model adaptability and reduces computational cost, which is valuable for real-world 3D segmentation scenarios, especially in cross-domain settings.\n2. Conceptual Novelty: The VIP module, inspired by multi-step reasoning (Chain-of-Thought), introduces a novel perspective to prototype refinement in few-shot segmentation.\n3. DyTaylorConv: The idea of incorporating Taylor series principles into convolution is interesting and adds interpretability to the modeling of local geometry. \n4. Strong Empirical Results: The method demonstrates consistent gains on two benchmarks, indicating good generalization and effective design.\n\nQUESTIONS:\n1. Could the authors provide inference-time cost (e.g., FPS or latency)? While the proposed pre-training-free design is claimed to offer faster deployment and efficiency, Table 6 only reports training-related timings. Would the authors be able to include inference speed or FPS comparisons to more directly support this claim?\n2. Can the authors clarify the reasoning behind the significantly larger parameter count? Table 6 shows that the proposed model introduces noticeably more parameters than other methods. Could the authors elaborate on the design trade-offs involved, and whether this affects scalability, memory usage, or real-time deployment?\n3. Have the authors considered evaluating under newer few-shot segmentation protocols? Recent method [1] proposes an more challenging settings for PC-FSS. Would the authors consider testing their method under such settings to further validate its generalization ability?\n[1] An Z, Sun G, Liu Y, et al. Rethinking few-shot 3d point cloud semantic segmentation[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2024: 3996-4006.\n\nLIMITATIONS:\n1. Lack of Inference Efficiency Evaluation：While the authors claim that the proposed pre-training-free design facilitates faster deployment and efficiency, Table 6 only reports pre-training time, episode training time, and total training time. However, it completely omits inference time or frame-per-second (FPS) metrics, which are crucial for evaluating actual runtime efficiency and supporting the claim of faster speed. Without this, the claim of efficiency superiority remains unsubstantiated.\n2. Clarify and Justify Model Size and Its Impact: According to Table 6, the proposed method introduces a noticeably larger number of parameters compared to existing approaches. The author should include a brief discussion or ablation to justify the design choice. Such clarification could also reinforce the fairness of the comparison and address potential concerns about scalability to larger or real-time scenarios.\n3. Missing Evaluation under Updated Few-Shot Settings: Recent works (e.g., [1]) have introduced more reasonable and challenging settings for few-shot point cloud segmentation. Including evaluations under such settings could further demonstrate the robustness and generalization capacity of the proposed approach. \n[1] An Z, Sun G, Liu Y, et al. Rethinking few-shot 3d point cloud semantic segmentation[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2024: 3996-4006.  \n4. To enhance the reproducibility and facilitate future research, it would be highly beneficial if the authors could consider releasing the source code and trained models. Additionally, providing some qualitative visualizations or the segmentation results would offer valuable insight into the behavior and limitations of the proposed method.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes VIP-Seg framework for Point Cloud Few-Shot Semantic Segmentation with the two major contributions with promising performance validated on S3DIS and ScanNet:\n1) VIP Module: Introduces a multi-step reasoning mechanism inspired by Chain-of-Thought, consisting of a Prototype Enhancement Module (PEM) and a Prototype Difference Module (PDM).\n2) DyTaylorConv: A Dynamic Taylor Convolution module that models local geometric structures via learnable polynomial approximations.\n\nSTRENGTHS AND WEAKNESSES:\nStrength:\n1) The paper is technically sound with clear mathematical formulation.\n2) Ablation studies and architectural design decisions are well-motivated and extensively analyzed \n3) Provides significant gains across datasets and settings \n\nWeakness: \n1) Although this paper tries to analogy to chain-of-thought reasoning, however I feel this work abuse the words to describe conventional prototype refinement and enhancement process in few-shot segmentation\n2) While training time is reduced, DyTaylorConv and the multi-step reasoning mechanism increase inference complexity and parameter count. No comparison of inference speed or resource usage is provided.\n3) Though cross-domain concerns are discussed, no explicit evaluation is performed on domain-shifted data (e.g., synthetic to real, indoor to outdoor).\n4) No justification is given for why 4 reasoning steps are optimal (aside from empirical observation).\n\nQUESTIONS:\nBesides the weakness, I may have other concerns on the technical aspect of this work:\n1. Inference Cost: What is the inference time or FLOPs for VIP-Seg compared to Seg-PN or other baselines?\n2. Domain Generalization: How does your method perform under cross-domain settings or on noisy/real-world sensor data?\n3. Ablation on PEM vs PDM Order: Would reversing the order (PDM-PEM) in the reasoning chain yield similar performance?\n\nLIMITATIONS:\nWhile the paper eliminates the need for pre-training, it introduces complex modules (e.g., DyTaylorConv, multi-step VIP reasoning, gating fusion), which increase inference-time computation. The paper discusses domain gaps and proposes PDM to address them, but does not evaluate on cross-domain benchmarks.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper tackles Point Cloud Few-Shot Semantic Segmentation by introducing VIP-Seg, a pre-training-free framework. Two key innovations are proposed: 1) Dynamic Taylor Convolution, which models local geometric structures via a learnable Taylor-series-inspired polynomial fitting, combining lower-order trigonometric encoding with dynamic higher-order experts, 2) Visual Introspective Prototype Module, which consists of alternating Prototype Enhancement Modules and Prototype Difference Modules in a multi-step process which seeks to address intra-semantic diversity and domain gaps. A gating fusion network then aggregates stepwise predictions. Extensive experiments on S3DIS and ScanNet demonstrate 5–10% absolute mIoU gains over prior work, while eliminating costly pre-training.\n\nSTRENGTHS AND WEAKNESSES:\n# Strengths\n1. The paper is well-structured and easy to understand.\n2. The experiments are quite comprehensive and shows that the proposed method outperforms the state-of-the-art approaches.\n\n# Weaknesses\n1. The authors claims that the Dynamic Taylor Convolution is inspired from Taylor Series. However, instead of a first and second order approximation of the pointcloud local geometry, the low-order and high-order convolution seem more like two distinct layer types with differing receptive fields. Furthermore, HiConv is not strictly looking at higher order geometric information but is a mixture of experts. As such, I find the naming confusing.\n2. Expanding on (1) above, there is insufficient discussion on other methods which consider differing receptive fields for point set convolution. This discussion should be added to motivate the authors' particular choice of DyTaylorConv. With that being said, the Taylor-series metaphor doesn't appear to be theoretically novel.\n3. I appreciate the provided computational efficiency analysis, but there is no inference-time latency or GPU-memory profiling provided, which leaves the real-time applicability unclear.\n4. VIP-Seg has over 10x the number of parameters as Seg-PN (2.77M params vs. 0.24M params). It's not clear if the performance improvements are due to the larger model size only and not the architectural improvements.\n5. The authors only conclude the limitation in one sentence in the main paper and discuss some very general limitations in the supplementary. This makes it difficult to understand when the method might fail.\n\n## Minor comments\n1. Performance depends on many settings (number of HiConv experts, reasoning steps S, gating weights, geometric priors) which seems clunky.\n2. Experiments results should not be listed as contributions.\n\nQUESTIONS:\n1. As mentioned above, can the author provide an more intuitive illustration or visualization on how the Dynamic Taylor Convolution relates to a Taylor Series? Or clarify further the use of this naming convention? See Weaknesses (1) and (2).\n2. In Section 3.3.4, the constructed weight $w_j$ is computed within HiConv, is this correct? In other words, $\\mathbf{W}_h$ is within HiConv?\n3. In Section 3.2, the authors mentioned Mamba blocks being added to the VIP module but I don’t see any description of how those blocks are added. Can you elaborate on this? \n4. The Prototype Enhancement Module resembles the Transformer architecture. Have you tried replacing it with Transformer? What would the performance differences be?\n5. Can you please address Weaknesses (3) and (4) given above, with regard to comparative inference times, GPU-memory profiling and network parameter numbers across methods?\n\nLIMITATIONS:\nNo. As mentioned in Weakness (5), the main paper doesn't have a dedicated limitations section and the supplementary contains a very general list of limitations.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}