GeoPurify: A Data-Efficient Geometric Distillation Framework for Open-Vocabulary 3D Segmentation
Authors: Weijia Dou
Xu Zhang
Yi Bin
Jian Liu
Bo Peng
Guoqing Wang
Yang Yang
Heng Tao Shen
Venue: ICLR 2026 Poster (2026)
Keywords: 3D Scene Understanding
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Abstract: Recent attempts to transfer features from 2D Vision–Language Models (VLMs) to 3D semantic segmentation expose a persistent trade-off. Directly projecting 2D features into 3D yields noisy and fragmented predictions, whereas enforcing geometric coherence necessitates costly training pipelines and large-scale, annotated 3D data. We argue that this limitation stems from the dominant \textit{segmentation-and-matching} paradigm, which fails to reconcile 2D semantics with 3D geometric structure. The geometric cues are not eliminated during the 2D-to-3D transfer but remain latent within the noisy and view-aggregated features. To exploit this property, we propose \textbf{GeoPurify} that applies a small Student Affinity Network to purify 2D VLM-generated 3D point features using geometric priors distilled from a 3D self-supervised teacher model. During inference, we devise a Geometry-Guided Pooling module to further denoise the point cloud and ensure the semantic and structural consistency. Benefiting from latent geometric information and the learned affinity network, GeoPurify effectively mitigates the trade-off and achieves superior data efficiency. Extensive experiments on major 3D benchmarks demonstrate that GeoPurify achieves or surpasses state-of-the-art performance while utilizing only \textbf{$\sim$1.5\%} of the training data.

Published as a conference paper at ICLR 2026
GEOPURIFY: A DATA-EFFICIEN GEOMETRIC DISTIL- LATION FRAMEWORK FOR OPEN- VOCABULYY 3D SEGMENTATION
Weijia Dou'
Xu Zhang"
Yi Bin
Jian Liu'
Bo Peng?
Guoqing Wang'
Yang Yang" Heng Tao Shen'
'Tongii University
'Tiajjn University
Uniieesitty of Electronic Science and Technology of China
Code Repository: https://githann ctt/t/12223////o/ooue
ABSTRACT
Recent attempts to transfer features from 2D Vision-Language Models (VLMs) 3D semantic segmentation expose persistent trade-off. Directly projecting 2D features into 3D yields noisy and fragmented predictions, whereas enforcing geo- metric coherence necessitates costly training pipelines and large-scale, annotated 3D data. We argue that this limitation stems from the dominant segmentation- and-matching paradigm, which fails to reconcile 2D semantics with 3D geomet- ric structure. The geometric cues are not eliminated during the 2D-to-3D trans- fer but remain latent within the noisy and hiew-aggregated features. To exploit this property.
propose GeoPurify that applies a small Student Affinity Net- work to purify 2D VL.M-generated 3D point features using geometric priors dis- tilled from a 3D self-supervised teacher model. During inference, we devise Geometry- Guided Pooling module to further denoise the point cloud and ensure the semantic and structural consistency. Benefiting from latent geometric informa- tion and the learned affinity network, GeoPurify effectively mitigates the trade-off and achieves superior data efficiency. Extensive experiments on major 3D bench- marks demonstrate that GeoPurify achieves or surpasses state-of-the perfor- mance while utilizing only ~1.5 of the training data.
INTRODUCTION
Efffective 3D scene understanding is critical for applications like autonomous driving, robotics, and augmented reality. However, progress hindered by the conventional closed-world paradigm, which assumes a fixed set of object categories. This approach fails to scale to the diverse and complex real-world objects and is further constrained by the prohibitive cost of manual 3D annotation, no- toriously laborious process that makes creating comprehensive datasets intractable (Dai al., 2017). To move beyond these limitations, the field is shifting toward -pen-vocabulary 3D understanding, which enables models to identify objects using arbitrary descriptions rather than predefined labels.
This new paradigm primarily leverages the broad semantic capacity of 2D Vision-Language Models (VLMM)) (Radford et al., 2021; Jia et al., 2021; Zou al., 2023) to extend limited set of semantic labels to open vocabulary. Existing approaches can be grouped into two categories: training free and training-based. Training- free methods directly exploit 2D VLMs for segmentation, projecting multi-view 2D predictions onto 3D point clouds, and merging them to obtain final outputs. How- ever, this often results in severe geometric inconsistencies, since 2D models lack awareness of 3D spatial structure. Training-base based methods, in contrast, attempt to learn lovet-level mappings from 3D geometry to semantic labels, but require dense point cloud annotations, which are costly and labor-intensive obtain. As result, current approaches are constrained by an unfavorable trade- off: either accept noisy and fragmented outputs from training-fre free projection or incur the heavy data and computational burden of training-bass based pipelines to enforee geometric coherence.
In the 2D-to-3D transfer, the 2D models typically follow "Segmentation and Matching" paradigm (Li et al., 2022; Ghiasi et al., 2022; Wang al., 2024a), a brittle two-step process that
"Corresponding author. Email: bin @ hotmail.com

---

Published as a conference paper at ICLR 2026
20 Distilled Fustures (Grometric Iiconslsteecc)
30 Goometric Features (Strectural Callerence)
Original
cdled
(n)
(b)
Figure 1: The Fundamental Disconnect: Semantic Richness vs. Geometric Coherence. Left: Original RGB 3D scene. Middle: Features distilled from 2D VLMs (Zou et al., 2023) offer rich se- manties but exhibit geometric inconsistency, leading fragmented and noisy segmentations. Right: Features from 3D self-supervised model (Wu et al., 2025) provide strong structural coherence with geometrically sound segments, yet inherently lack open-vocabulary semantic understanding. Our proposed method aims to bridge this critical gap by purifying the semantically rich 2D features with robust 3D geometric priors.
disconnects geometry and semantics. We argue that machine perception should instead follow "Segmentation as Understanding" paradigm, where recognizing an object's form and understanding its meaning intrinsically linked. However, simply adopting generalist VLM that follows this paradigm is insufficient. As shown in Figure 1-(b), 2D VLM features (Faem) are semantically rich but geometrically inconsistent, resulting fragments and shape distortion, whereas priors from 3D) self-supervised models gGooo) are geometrically reliable but lack open-vocabulary semantics, as il- lustrated in Figure 1-(c). Inspired by Wysoczańška et al. (2024), we hypothesize that transferring 21) features into 3D does not destroy the geometric information, but rather renders it latent, suggesting that we may recover latent structure more efficiently instead of learning 3D geometry from scratch.
Motivated by this hypothesis, we present GeoPurify, a data-efficient framework designed to re- cover latent geometric structure from noisy semantic features and produce robust 3D representa- tions. We first leverage one-for-all VLM to generate semantic-rich and geometric-aware 3D fea- tures from multiple 2D views. To address the noise and fragmentation inherent -iee-aeee point clouds, we propose Geometric Contrastive Distillation, where student module learns la- tent geometric affinitics from noisy features under the guidance of a 3D teacher's s structural priors. By exploiting the pre-existing geometric knowledge, GeoPurifyy eliminates the need for large-scale manual annotation and can be trained using only a small set unlabeled 3D scans. At inference, Geometry- -uidedd Pooling module is designed to denoise and consolidate the features, producing unified 3D representations that are both semantically rich and geometrically coherent. Experiments on multiple popular 3D semantic segmentation datasets demonstrate that GeoPurify achieves com- parable or superior performance to state-of-the-ar methods while using only ~ 1.5 % of the training data, effectively resolving the trade-off that limits prior approaches.
In summary, our contributions are:
We introduce GeoPurify, data-efficient framework built on the hypothesis that beyond also embed a latent 3D geometric struc- ture. GeoPurify demonstrates that recovering this latent structure is more data-efficient path to open-vocabulary 3D segmentation.
their semantic richness, VLM MLo--erjected features
Our framework first leverages a generalist VLM to establish rich semantic foundation via "Segmentation as Understanding". overcoming the low semantic ceiling of prior "Segmen- tation and Matching" approaches It then introduces a Geometric Contrastive Distillation mechanism to learn latent geometric affinities from unlabeled 3D scans and a Geometry- Guided Pooling module that uses these affinities to denoise features and ensure structural consistency at inference.
Extensive experiments on multiple 3D benchmarks show that GeoPurify achieves perfor- mance comparable to better than state-of-the-art methods while using only ~1.5% of the training data, highlighting its superior data efficiency. To ensure reproducibility and foster future research, we will publicly release code and pre-trained models.
2

---

Published as a conference paper at ICLR 2026
2 RELATED WORK
Closed-Set 3D Scene Understanding Closed-set 3D scene understanding, which assumes fixed vocabulary of object categories, is dominated by two paradigms: Voxel-based and Point-based meth- ods (Guo et al., 2020). Voxel-based approaches discretize scenes into grids and leverage sparse con- volutions for computational efficiency (Graham al., 2018; Choy et al., 2019; Zhao et al., 2023). Point-based methods, in contrast, operate directly on raw point clouds to preserve geometric detail, evolving from the pioneering PointNet(+++) (Qi et al., 2017a;b) to modern local aggregation oper- ators (Xu et al., 2021; Wu et al., 2019) and efficient Transformer architectures (Duan et al., 2023; Liu et al., 2023; Wu et al., 2024). While highly successful on benchmarks like ScanNet (Dai et al., 2017) and S3DIS (Armeni et al., 2016), their foundational closed-set assumption prevents general- ization to unseen categories, critically limiting real-world applicability and motivating the shift to open-vocabulary models.
Open-Vocabule Vocabulary 3D Scene Understanding To overcome closed-set limitations, open- vocabulary 3D scene understanding methods transfer rich semanties from 2D Vision-Language Models (VLMs). The core challenge is rectifying geometric inconsistencies that arise when pro- jecting multi-view 2D features into a unified 3D space. Current solutions often tackle this with extensive supervision. Some perform simple training-free feature aggregation, but this yields noisy and fragmented 3D features (Jatavallabhula et al., 2023; Lu et al., 2023; Peng et al., 2023; Nguyen et al., 2024). While subsequent pesse-socessing methods like PoVo (Mei et al., 2025) attempt to refine these initial features, they often fail to enforce consistency for complex or occluded objects. The dominant approach is large-scale knowledge distillation, where 31) backbone is trained on noisy pseudo-labels from 2D projections, requiring massive datasets to implicitly learn geometric priors (Chen al., 2023b; Peng et al., 2023; Li et al., 2024; Wang et al., 2024c; Li et al., 2025; Wang et al., 2024b; Xu et al., 2024). Others attempt direct 3D-text alignment, but typically rely on coarse, region-level text supervision that lacks the granularity for dense prediction tasks (Ding et al., 2023; Yang et al., 2024; Wang et al., 2025; Jiang et al., 2024). Recent works such as OV3D (Jiang et al., 2024) and PGOV3D (Zhang et al., 2025) advance VLM-based understanding pipelines but typically rely on large-scale data for alignment or curriculum learning, missing the efficiency of explicitly recovering latent geometric affinities. Ultimately, while techniques like AFOV (Sun et al., 2025) enhance annotation efficiency, they still share fundamental reliance massive datasets to implic- itly resolve geometric ambiguities through improved supervision. This reveals a critical gap for a data-efficient method that explicitly enforces geometric coherence during the 2D-to-3D semantic transfer.
3 METHODOLOGY
3.1 OVERALL ARCHITECTURE
As illustrated in Figure 2, our proposed GeoPurify first leverages frozen Vision-Language Model (p2p) to transfer and merge multi-view RGB images into an initial 3D feature map F.om E R. N Although semantically rich, these features lack local geometric coherence. To address this, we introduce 3D student affinity network Øs that learns geometric affinities from the point cloud using a self-supervised 3D geometric model to refine Fsem. The student Øs is trained with- out semantic labels through knowledge distillation from frozen 3D self-supervised teacher Φr. Specifically, Φs learns to recover the latent geometric structure embedded in Fsem and approximate the teacher's 's geometric representation by minimizing InfoNCE contrastive loss over point triplets sampled from ΦT 's embedding space. At inference, the semantic-rich and geometry-awan features further refined by single geometric pooling operation, which enforces local consistency through information aggregation among geometrically similar points.
3.2 SEMANTIC INITIALIZATION FROM A GENERALIST VLM
To obtain 3D representations enriched with semantic priors, we project RGB inputs into the 3D point space (constructed by aggregating multi-view projections, without necessitating external raw point clouds at inference) using a frozen 2D VLM and merge multiple views to capture structural cues. Prior methods often follow Segmentation and Matching paradigm, as in LSeg (Li et al., 2022),
3

---

Published as a conference paper at ICLR 2026
2D VLM 1
Initaal 3D Featunes
Multi-view RGB images
Predicied
&s Student A Affinity Network
Geometric himheddings
Contrastive
Distillation
Geometric Pooling
AD Geomettic SSL OT
Geometric Embeddings
Refined 3D
Features
3D Point Cloud P gxs
Only Trinn Phase Train Infer Phase
Figure 2: GeoPurify: A Data-Efficient Pipeline for Geometric Purification of 3D Semantic Features. Our method consists of two stages. 1) Training (left, dotted path): A Student Affinity Network (ds) is trained to comprehend 3D structure. learns geometric relationships directly from the point cloud, using contrastive distillation to mimic the embeddings of powerful, frozen 3D) SSL teacher (ØT). This training phase requires no 3D semantic labels. 2) Inference (right, solid path): A frozen 2D VLM (V22)) generates initial 3D features by projecting rich semantic content from multi-view images. These features, however, are geometrically inconsistent. The pre-trained student network then applies a geometry-aware pooling, using its learned affinities refine the initial features. This process yields final representation that is both semantically rich and geometrically coherent.
OpenSeg (Ghiasi al., 2022), and SAM+CLIP frameworks (Wang et al., 2024a), which adopt Lo- calize then Recognize pipeline. They first group pixels into regions using segmentation backbones and then match these clusters to predefined approach produces highly discriminative yet task-specific features, limiting their semantic ceiling and reducing their capacity to capture the generative and associative richness needed open-world 3D environments. To overcome this limitation, we adopt X-Decoder (Zou et al., 2023) as frozen 2D VLM backbone (V)) Instead of isolating segmentation as a downstream task, X-Decoder con- structs unified visual-linguistic embedding space that supports diverse tasks such segmentation, retrieval, and dense captioning. Its dual-query architecture, which learns from both latent visual queries and explicit text queries, makes segmentation
labels. While effective in closed-set settings, this
emergent property of deeper contextual understanding. The resulting feature space is inherently generative and associative, optimized to answer the holistic question: What is depicted, where is it located, and how can it be described in language? These richer representations provide higher semantic ceiling and yield stronger 21) priors for 3D scene understanding.
Follow this pipeline, to produce 3D point cloud P P= {pr) with its corresponding multi-view images {T, V= we first compute dense feature maps for each image using 22n. For each point its feature fr,v sampled from the feature map of each visible view via camera projection T.. The features from multiple views are then aggregated using weighted averaging scheme to produce single initial semantic descriptor from. This yields the initial feature set
Fsem
(
N i=
ND
Although semantically potent, this feature aggregation from 2D views introduces geometric incon- sistencies, necessitating the subsequent purification stage detailed next.
3.3 GEOMETRIC CONTRASTIVE DISTILLATION
To rectify the geometric inconsistencies within the initial semantic features F Fsems we introduce contrastive purification module trained via knowledge distillation. This framework employs
4

---

Published as a conference paper at ICLR 2026
powerful, frozen 3D foundation model, Sonata (Wu al., 2025), the teacher (δT) to provide a robust geometric target space, and a trainable sparse 3D CNN as the student (os). Crucially, the student's objective not to replicate the teacher's features but to learn the geometric affinities between points as defined by the teacher's embedding space. The student network therefore operates directly the point cloud geometry P, learning to produce geometrically-awe embedding space that will later be used to refine Fsem-
The knowledge transfer is driven by contrastive objective structured via an efficient hybrid sam- pling strategy that distills both global and local context from the teacher's feature space. For each anchor point Pa. we define its positive pair Pp as the point with the highest feature similarity across the entire scene. We then curate set of hard negatives comprising two distinct types: macro- negatives, which are the points globally most dissimilar to Pa the feature space, teaching the model overall scene structure; and micro-negatives, which are sampled from Pa 's spatial neighbor- hood but are specifically chosen for having the lowest feature similarity within that local region. This micro-sampling of spatially proximate yet featurally distant points forces the model to learn fine-grained geometric distinctions. The student network, Øg, which maps the point cloud P to set of geometric embeddings Ggeo R Paco, is then optimized to organize its embedding space according to this distilled affinity information by minimizing the InfoNCE loss:
exp(sim(ga. 9p)/T)
== - log
(2)
exp(sim(g..9)) 9,)/r) + Σσ_ exp(sim((s.).))))
where g E Ggeo are the geometric embeddings produced by the student ds, sim(::) is the cosine similarity, and T is a temperature hyperparameter. We specifically choose a contrastive objective over direct feature regression, as our goal is not to replicate the teacher's feature vectors but to distill its underlying relational structure. By optimizing this objective, és learns encode the intrinsic geometric structure of the scene, producing representation capable of enforcing structural coherence upon the initial semantic features.
3.4 PURIFYING FEATURES VIA GHOMETRY-GUIDER POOLING
At inference, the Geometry- Guided Pooling module refines the initial semantic features Frem. The trained student network ós first processes the voxelized point cloud to generate a geometric embed- ding 9s for each of the V active voxels. From these embeddings, A E I XV that encodes local geometric relationships. For each voxel u; we identify its K nearest spatial neighbors form local neighborhood. (i). The affinity Ai neighbor U/ A(i)) is computed by applying a sharpened softmax to the cosine similarity of their respective embeddings:
construct sparse affinity matrix
exp(a sim(g. g;))
Aij ΣkeN(a) exp(xx sim(gs,
where sim(..) -) is the cosine similarity and a is a sharpening factor, analogous to an inverse temper- ature, that controls the concentration of the affinity weights.
The resulting matrix A guides an iterative pooling process that propagates semantic information across the learned geometric graph. Initializing with the voxel-aggregated semantic features p(O), update the features for T steps:
(t-1)) = AF
(4)
Finally, the refined voxel features F(T) are mapped back to their corresponding original points, yielding the purified feature set Fi.m- This iterative refinement enforces local geometric consis- tency by averaging features based on the learned structural affinities, effectively denoising the initial representation while preserving its semantic richness.
4 EXPERIMENTS
4.1 EXPERIMENTAL. SETUP
Datasets and Data-Efficient Training. We evaluate our method on three prominent 3D indoor scene understanding benchmarks. Our primary evaluation is conducted on ScanNetV2 (Dai al.,
5

---

Published as a conference paper at ICLR 2026
2017), large-scale dataset of over 1,500 RGB-D scans from diverse indoor environments, and Matterport3D (Chang et al., 2017), which contains 90 building visual detail. For a rigorous analysis of performance muileng--saae object categories, we also leverage the challenging long-tail benchmark ScanNer200 (Rozenberszki et al., 2022).
scale scenes with rich geometric and
As aforementioned, benefiting from the geometric guidance distillation recover the latent struc- ture information, our GeoPurify could be learnt efficiently. We sample a highly compact subset of original training data, comprising merely 20 scenes (~1.6%) from ScanNetV2 and 20 scene regions (~1.3% ).3%) from Matterport3D. Once this compact subset is selected, the subsequent dis- tillation training is performed without using any 3D semantic labels, relying only on the raw point cloud geometry and multi-view imagery of these 20 scenes. To make the subset contain more object semantics, we leverage the ground-trute semantic statistics of the full training set to identify the subset with maximum variance. We posit that the most informative scenes are not simply those with many object types, but those where these objects appear balanced proportions.
To formalize this, our selection process is guided by two metrics. The first, semantic richness (No), a direct count of unique object categories. However, richness alone is insufficient, as a scene can have many categories yet be dominated by trivial structures like walls and floors. We therefore in- troduce a more critical metric, semantic complexity (He), measured via Shannon entropy (Shannon, 1948):
He -1=-Erie p(c) loga p(c), OEC
(5)
where p(c) is the proportion of points in category c. Entropy directly quantifies the scene's infor- mational density. A high-entropy scene, such as a cluttered office, exhibits a balanced distribution of diverse objects and thus offers a rich learning signal. Conversely, a low-entrppy scene, like an empty hallway, is predictable and provides limited training value.
Our selection pipeline unfolds in three stages. First, we filter for quality, culling any scene that falls below the median value for both richness (N₂) and complexity (H₂). Second, to ensure the subset spans diverse environmental archetypes (e.g.. kitchens, offices), we perform K-Means clustering on the semantic category histograms of the filtered scenes. Finally, from each resulting cluster, we select the single most exemplary scene by ranking them with a composite score,
= Henorrm + y Nemnorm:
(6)
which jointly rewards normalized complexity and richness. This strategy yields a training set that not merely small but is curated distillation of the dataset's core semantic diversity.
Evaluation Metrics. We evaluate our method on the task of open-vocabulary 3D semantic seg- mentation, focusing the zero-shot setting. We follow the evaluation protocol established in CUA- O3D (Li et al., 2025) and report mean Intersection -oroon--ov---nnion (mloU) and mean Accuracy (mAcc). Furthermore, to assess performance benchmark, we also report foreground mloU (f-mloU) and foreground mAce (f-mAcc). Following established practice (Ding et al., 2023; Jiang et al., 2024; Yang et al., 2024; Wang et al., 2025), these foreground metrics are calculated on all classes while excluding common structural categories like "wall", "floor", and "ceiling".
wider range of object categories from the ScanNer200
Implementation Details. We use pre-trained X-Decoder (Zou et al., 2023) (DaViT-L) as our frozen 2D semantic backbone (V'22) and the frozen Sonata (Wu et al., 2025) model our geometric teacher (Φr). Our student network (øs) is a sparse 3D network built with a series of residual blocks that produces a 22-diimnsioonal geometric embedding for each point. The student network is trained for 50 epochs using the AdamW optimizer with a learning rate of 1 x 10 10-3, which is decayed using a cosine annealing schedule. For the InfoNCE loss, the temperature 7 is set to 0.07. During training, we sample 4096 anchor points per scene and, for each anchor, we sample 64 negatives (48 macro, 16 micro). At inference, the Geometry- Guided Pooling is applied for = 18 iterations with an affinity sharpening factor set to 1/20.0. All experiments are conducted on single NVIDIA L40 GPU.
4.2 QUANTITATIVE RESULTS
We evaluate GeoPurify against sfate-of-the-an methods on open- voca-uucy and long-tail 3D seg- mentation benchmarks. The results demonstrate that by distilling geometric prior from a small data

---

Published as a conference paper at ICLR 2026
subset (~1.5%), GeoPurify achieves competitive, and in some cases state-of-the-an performance without requiring large-scale 3D semantic annotations.
Open- Veee-Vocay Segmentation Performance. As shown in Table GeoPurify demonstrates exceptional performance by leveraging a curated data subset of only ~1.5%. To rigorously validate that our method's efficacy stems from its architecture rather than just the data selection, we re- trained a leading competitor, CUA-O3D (Li et al., 2025), on the exact same compact subset. Under these identical data-efficient conditions, CUA-O3D's performance drops sharply to an 18.1 mloU on ScanNerV2. In stark contrast, GeoPurify achieves an mloU of 55.1. We attribute this robustness to our decoupled design. Standard methods typically attempt to learn entangled geo-semantic rep- resentations from scratch, failing to generalize without sufficient data. In contrast, GeoPurify relies on the frozen VLM for semantics and trains the student network solely to recover latent geomet- ric structure. Learning these geometric grouping rules requires significantly less data, allowing our method to function as an efficient "geometric denoiser" that maintains high performance.
Remarkably, model not only excels in this low-data regime but also rivals methods trained on the full dataset. The advantage is even more pronounced in the mean Accuracy (mAce) metric, where GeoPurify attains 72.5 on ScanNetV2 and 62.4 on Matterport3D. substantially outperforming fully- trained competitors. This divergence between high mAce and competitive mloU is an intentional outcome of our design. The superior mAce is driven by Geometry- agates VLM-initialized core labels across geometrically coherent regions, enforcing intra-object consistency and boosting recall. This mechanism, however, introduces a deliberate trade-off: by prioritizing semantic purity within objects, it can cause minor "semantic bleeding" at boundaries, slightly degrading the boundary precision to which mloU is sensitive. For a detailed qualitative analysis, including visual examples, please refer to Appendix D.
Pooling, which prop-
Table I: Quantitative results for open-vocabulary 3D semantic segmentation on ScanNetV2 and Matterport3D. We compare GeoPurify, trained with only ~1.5% of the 3D data, against fully supervised (Fully-sup.) and zero-shot (Zero-shot) methods that utilize the full training set. Values in bold indicate the best result for each metric among the Zero-shot and Data-Efficient Zero-shot methods. 'Denttes results from the original paper's LSeg-based implementation. DDnootes results reproduced by CUA-03D (Li et al., 2025) and MPEC (Wang et al., 2025).
3D Net Training Data
Type
Method
ScanNetV2
Matterport3D
mloU
mAce mloU
mAce
TextureNet Huang al. (2019)
100%
54.8
63.0
ScanComplete (Dai et al., 2018)
100%
56.6
44,9
DCM-Net (Schult et al., 2020)
100%
65.8
66.2
SupCon (Zheng et al., 2021)
100%
69.2
77.7
53.1
63.4
Fully-sup.
MinkowskiNet (Choy et al., 2019)
100%
69.2
77.7
53.1
63.4
LGround (Rozenberszki et al., 2022)
100%
73.2
67.2
PLA (Ding al., 2023)
100%
17.7
33.5
CLIP2Scene (Chen al., 2023b)
100%
25.1
CNS (Chen et al., 2023a)
100%
26.8
CLIP-FO3D (Zhang et al., 2023)
100%
30.2
49.1
RegionPLCC (Yang al., 2024)
100%
43.8
65.6
28.9
43.8
MSeg Voting (Lambert al., 2020)
100%
45.6
54.4
33,4
Zero-shot
DMA-text only (Li et al., 2024)
100%
50.5
63.7
39.8
49.5
DMAf-3D (Li et al., 2024)
100%
54.8
66.9
OpenScene 11.3D (Peng et al., 2023)
100%
51.6
63.1
40.5
48.8
CUA-O3D (3D) (Li et al., 2025)
100%
54.1
64.1
41.3
49.5
GGSD (Wang ct al., 2024b)
100%
56.5
68.6
OV3D (Jiang et al., 2024)
100%
57.3
72.9
45,8
62.4
PGOV3D (Zhang al., 2025)
100%
59.5
73.2
CUA-O3Desimple (3D) (Li et al., 2025)
~ 1.55%
18.1
26.4
14.0
20.5
Duta-E Elfac-fffiient Zero-shot
(Ours) GeoPurify
~ 1-.5%
55.1
72.5
40.2
62.4
Generalizability on Long-Tail Datasets. As shown in Table 2, GeoPurify establishes new state- of-the-art on long-tail benchmarks like ScanNet200 and the challenging M160 split. This robust generalization arises from the synergy between semantic and geometric modules. Our generalist.

---

Published as a conference paper at ICLR 2026
VLM provides high semantic ceiling, generating descriptive "semantic seeds" even for ob- jects where traditional iengmitiionfooussee backbones might fail. These initial, often sparse, signals then amplified by our class-agnostic geometric prior. Because this prior is distilled purely from unlabeled geometry, it is immune to the frequency bias inherent in semantic datasets. During infer- ence, the Geometry- Guided Pooling leverages this unbiased structural knowledge propagate the VLM's semantic seeds across entire geometrically coherent instances. In short, the VLM proposes weak semantic hypothesis, and our geometry-aware network confidently validates and completes it, ensuring robust segmentation even for categories with scarce data.
Table 2: Zero-shot segmentation performance on long-tail benchmarks. We report key metrics (f-mloU, f-mAcc) on ScanNer200 and (mIoU, mAcc) on frequeney-based splits of Matterport3D (top K=40, 80, 160 classes). Our data-efficient GeoPurify is compared against other zero-shot base- lines. Values in bold indicate the best result for each metric among methods. RResults from the orig- inal LSeg-based implementation. *Results reproduced by DMA (Li al., 2024) and MPEC (Wang
et al., 2025).
Method
ScanNet200
Matterport40 mloU mAce
Matterport80 mAcc
Matterport160 mAce
f-mloU
f-mAcc
mlaU
mloU
PLA (Ding al., 2023) DMA-text only (Li al., 2024)
1.8
3.1
6.9
11.3
25.4
31.6
11.7
16.1
6.2
8.0
DMA(FC-CLiPP--D) (Li ct al., 2024)
7.9
15.2
38.4
48.3
20.1
26.5
9.8
15.2
OpenScene (Peng et al.. 2023)
7.3
25.4
30.7
12.0
15.2
5,9
7.5
RegionPLC (Yang et al., 2024)
9.1
17.3
(Ours) GcoPurify
11.9
22.8
33.1
48.0
18.6
31.7
11.9
20.5
Cross-Datasel Generalization. We evaluate zero-shot cross-datased generalization to test the ro- bustness of our learned geometric prior. As shown in Table 3, GeoPurify significantly outperforms existing methods in both transfer directions. The advantage is particularly pronounced when trans- ferring from Matterport3D >> ScanNetV2, where our method achieves 54.9 mloU, surpassing the next-best competitor by large margin of 16.3 points.
This striking performance gap reveals a fundamental difference what is learned. While Matter- port3D's scenes are geometrically rich, methods that learn entangled geo-semantic representations tend to overfit to this richness. Their learned priors become correlated with Matterport3D's specific object styles and semantic distribution, making them brittle when transferred to domain like Ve. In contrast, our decoupled training process allows GeoPurify to capitalize on the geo- metric complexity without semantic overfitting. By distilling knowledge based purely on structural affinities, our student network learns class-agnostic, geometric prior that captures abstract shapes and relationships. This purely structural knowledge is highly transferable, acting as a robust regular- izer even when the semantic context changes entirely. This result provides compelling evidence that our method learns a more fundamental and truly domain-agnostin understanding of 3D geometry. A more granular analysis of cross-domain generalization under an expanded vocabulary is presented in Appendix B.2.
Table 3: Zero-shot cross-dataset evaluation. Models are trained on the source dataset and evalu- ated directly on the target without fine-tuning.
ScanNelV2 mloU (%)
Matterport3D mAce (%)
Matterpori3D + Scan mloU (%)
Method
mAce
OpenScene (Peng al., 2023)
36.0
48.0
36.5
44.0
CUA-O3D (Li et al., 2025)
37.4
49.2
38.6
46.6
GGSD (Wang et al., 2024b)
40.1
54.4
GeoPurify (Ours)
40.5
62.7
$4.9
71.9
4.3 ABLATION STUDIES
We conduct series of ablation studies on the ScanNetV2 validation set to analyze the contribution of each component within our GeoPurify framework. The results, summarized in Table 4, system- atically deconstruct model validate its core design principles.
8

---

Published as a conference paper at ICLR 2026
Table 4: Ablation studies of GeoPurify the ScanNetV2 validation We analyze the impact
of our core geometric purification module, the choice of 2D backbone, the contrastive sampling strategy, the number pooling iterations (I'), and the training subset size. Our default configuration is highlighted in bold.
Component
Setting
mIoU
mAcc
w/o Purification (Aggregated 2D features) GeoPurify (Ours)
50.2
68.1
Geometric Purification
55.1
72.5
LSeg LSeg + GeoPurify
48.6
61.6
2D Semantic Backbone
51.2
63.0
Macro-only Hybrit Aybrid (Ours)
53.5
70.8
Contrastive Sampling
55.1
72.5
T =
52.3
70.2
T
53.9
71.4
Pooling Iterations (T)
T = 18 (Ours)
55.1
72.5
T 36
55.1
72.4
scenes
54.7
72.4
20 scenes (Ours)
55.1
72.5
Training Subset Size
30 scenes
55.1
72.5
50 scenes
55.0
72.5
Geometric Purification We first validate the efficacy of our core module. A baseline that directly aggregates 2D features from the X-Decoder backbone without any geometric refinement achieves 50.2 mloU. The introduction of our GeoPurify module yields a significant performance increase 55.1 mloU (+4.9 mloU). This substantial gain underscores our central hypothesis: even with powerful semantic backbone, projected features suffer from geometric fragmentation, and an explicit purification step is essential to resolve these inconsistencies.
2D Semantic Backbone We next analyze the choice of the 2D backbone. While applying GeoPu- rify to a standard LSeg backbone achieves a respectable 51.2 mloU, replacing it with the generalist X-Decoder boosts performance to 55.1 mloU. This +3.9 mloU improvement demonstrates that the richer, more associative feature space the X-Decoder provides a superior semantic substrate for the subsequent geometric refinement process.
Hybrid Contrastive Sampling To dissect the purification module, we evaluate our sampling strat- egy. A variant trained using only neggttvvs results in a 1.6 mloU performance drop to 53.5 mloU. This confirms that micro-negatives are the primary mechanism for resolving local geometric ambiguities at object boundaries (e.g., diffcrentiating chair leg from the floor). Without them, the model Iearns the global scene layout but fails to disentangle co-located surfaces.
Pooling Iterations (7) Finally, we analyze the number of pooling iterations, T, which directly controls the receptive field of the feature refinement the geometric graph. Each iteration F(++1) AF(t) propagates information by hop, meaning a point's feature after T steps is influenced by its neighbors up to T hops away. As shown in Table 4, performance steadily improves from T (52.3 mloU) T 18 (55.1 mloU). This suggests that sufficiently large receptive field is necessary to propagate semantic information across entire coherent surfaces, thereby averag- ing initial projection noise. However, increasing the iterations further to T 36 leads to slight performance degradation. This indicates the onset of over- wwo-thno,,,ng, well-known phenomenon where an excessively large receptive field begins to merge features from distinct but adjacent objects, blurring crucial semantic boundaries. Thus, T = 18 represents optimal trade-off, maximizing information propagation within geometrically consistent regions while preserving the semantic dis- tinctiveness between them.
Training Subset Size We evaluate the method's 's sensitivity to data quantity by training on subsets of 10, 20, 30, and 50 scenes. As shown in Table 4, performance improves markedly from 10 to 20

---

Published as a conference paper at ICLR 2026
scenes but plateaus thereafter. This trend validates design principle: unlike methods learning complex semantic mappings, our student network distills class- -ansssinootic geometric affinities (e.g., surface contimuity) which are repetitive and universal. Consequently, small subset (20 scenes) provides sufficient structural diversity for convergence. 'he observed plateau indicates that our framework successfully recovers the latent structure available refine the current VLM features, suggesting performance is primarily bounded by the VLM's semantic quality rather than the quantity of geometric training data.
CONCLUSION
We introduce GeoPurify, a novel framework that resolves the fundamental conflict between seman- tic richness and geometric coherence open-vocabulary 3D scene understanding. We posit that the prevailing "segmentation and matching" paradigm is a primary bottleneck and propose a con- ceptual shift towards "segmentation as understanding." GeoPurify materializes this vision through data-efficient teacher-student architecture that distills latent geometric structure from noisy, view- aggregated VLM features. By leveraging Geometric Contrastive Distillation on unlabeled 3D scans and enforcing spatial consistency with Geometry-Guiee Guideed Pooling at inference, our method effec- tively reconciles the two modalities. Extensive experiments show that GeoPurify achieves state-of- the-art or competitive performance major 3D benchmarks without any 3D annotations, highlight- ing its potential as scalable, foundational approach for annotation-free 3D perception.
ACKNOWLEDGMENTS
This work is supported by the Central Guidance on Local Science and Technology Development Fund Shanghai City (YDZX20253) 100002004), National Key Research and Development Pro- gram of China (2055YFF0222500) the Fundamental and Interdisciplinary Disciplines Breakthrough Plan the Ministry of Education of China (YYB2055DDXM166) and the Fundamental Research Funds for the Central Universities.
ETHICS STATEMENT
Our research utilizes publicly available academic benchmarks (ScanNetV2, Matterport3D, Scan- Net200) and adheres to their standard usage protocols. We acknowledge that these datasets may contain geographic and cultural biases, which could impact model fairness and generalization across diverse real-world environments. While the technology is intended for beneficial applications like robotics and augmented reality, we recognize the dual-use potential inherent to all advanced percep- tion systems.
REPRODUCIBILITY STATEMENT
To ensure our work is reproducible, we will publicly release all code and pre-trained models. The core methodology is detailed in Section 3, with evaluation protocols specified in Section 4.1. Ap- pendix offers a complete implementation guide, detailing network architectures, the exact data subset selection logic, and comprehensive hyperparameter table.
REFERENCES
Iro Armeni, Ozan Scner, Amir R Zamir, Helen Jiang, Ioannis Brilakis, Martin Fischer, and Silvio Savarese. 3d semantic parsing of large-scale indoor spaces. In Proceedings the IEEE conference on computer vision and pattern recognition, pp. 1534-1543, 2016.
Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Halber, Matthias Niessner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang. Matterport3D: Learning from RGB-D data in indoor environments. International Conference on Vision (3DV), 2017.
Rumnan Chen, Youquan Liu, Lingdong Kong, Nenglun Chen, Xinge Zhu, Yuexin Ma, Tongliang Liu, and Wenping Wang. Towards fibel-free scene understanding by vision foundation models. Advances in Neural Information Processing Systems, 36:75896-75910, 2023a.
10

---

Published as a conference paper at ICLR 2026
Rumnan Chen, Youquan Liu, Lingdong Kong, Xinge Zhu, Yuexin Ma, Yikang Li, Yuenan Hou, Yu Qiao, and Wenping Wang. Clip2scene: Towards label-efficient 3d scene understanding by clip. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7020-7030, 2023b.
Christopher Choy, Jun Young Gwak, and Silvio Savarese. 4d spatio-temporal convnets: Minkowski convolutional neural networks. In Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, pp. 3075-3084, 2019.
Angela Dai, Angel X Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 5828-5833 2017.
Angela Dai, Daniel Ritchie, Martin Bokeloh, Scott Reed, Jürgen Sturm, and Matthias NieBner. Scan- complete: Large-scale scene completion and semantic segmentation for 3d scans. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4578 4587, 2018.
Runyu Ding, Jihan Yang, Chuhui Xue, Wenqing Zhang. Song Bai, and Xiaojuan Qi. Pla: Language- driven open-vocabulary 3d scene understanding. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 7010-7019, 2023.
Lunhao Duan, Shanshan Zhao, Nan Xue, Mingming Gong, Gui-Song Xia, and Dacheng Tao. Condaformer: Disassembled transformer with local structure enhancement for 3d point cloud understanding. Advances in Neural Information Processing Systems, 36:23886-23901, 2023.
Golnaz Ghiasi, Xiuye Gu, Yin Cui, and Tsung Yi Lin. Scaling open-vocabulary image segmentation with image-level labels. In European conference on computer vision, pp. 540-557. Springer, 2022.
Benjamin Graham, Martin Engeleke, and Laurens Van Der Maaten. 3d semantic segmentation with submanifold sparse convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 9224-9232, 2018.
Yulan Guo, Hanyun Wang, Qingyong Hu, Hao Liu, Li Liu, and Mohammed Bennamoun. Deep learning for 3d point clouds: A survey. IEEE transactions on pattern analysis and machine intelligence, 43(12):4338-453 2020.
Jingwei Huang, Haotian Zhang, Li Yi, Thomas Funkhouser, Matthias Nießner, and Leonidas Guibas. Texturenet: Consistent local parametrizations for learning from high-resolution signals on meshes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition, pp. 4440-4449, 2019.
Krishna Murthy Jatavallabhula, Alihusein Kuwajerwala, Qiao Gu, Mohd Omama, Tao Shuang Li, Ganesh Iyer, Soroush Saryazdi, Nikhil Keetha, Ayush Tewari, Joshua B. Tenenbaum, Celso Miguel de Melo, Madhava Krishna, Liam Paull, Florian Shkurti, and Antonio Torralba. Concept- fusion: Open-set multimodal 3d mapping. Robotics: Science and Systems (RSS), 2023.
Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representation learning with noisy text supervision. In International conference machine learning, pp. 4904-4916. PMLR, 2021.
Li Jiang, Shaoshuai Shi, and Bernt Schiele. Open-vocabulary 3d semantic segmentation with foun- dation models. In Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition, pp. 21284-21294, 2024.
John Lambert, Zhuang Liu, Ozan Sener, James Hays, and Vladlen Koltun. Mseg: A composite dataset for dolti- semantic segmentation. In Proceedings of the IEFF/CVF conference on computer vision and pattern recognition, pp. 2879-2888, 2020.
Boyi Li, Kilian Q Weinberger, Serge Belongie, Vladlen Koltun, and Rene Ranftl. Language-driven semantic segmentation. In International Conference on Learning Representations, 2022. URL https: https://openrevieee net forum? fooommiiddiaa dddeeiiiddddidddde

---

Published as a conference paper at ICLR 2026
Jinlong Li, Cristiano Saltori, Fabio Poiesi, and Nicu Sebe. Cross- and uncertainty aware agglomeration for open-vocabulay 3d scene understanding. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 19390-1944900 2025.
Ruihuang Li, Zhengqiang Zhang, Chenhang He, Zhiyuan Ma, Vishal M Patel, and Lei Zhang. Dense multimodal aligmment for open-vocabulary 3d scene understanding. In European Conference on Computer Vision, pp. 416-434. Springer, 2024.
Zhijian Liu, Xinyu Yang, Haotian Tang, Shang Yang, and Song Han. Flatformer: Flattened window attention for efficient point cloud transformer. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 1200-1211, 2023.
Shiyang Lu, Haonan Chang, Eric Pu Jing, Abdeslam Boularias, and Kostas Bekris. Ovir-3d: Open- vocabulary 3d instance retrieval without training on 3d data. In Conference on Robot Learning, pp. 1610-1620. PMLR, 2023.
Guofeng Mei, Luigi Riz, Yiming Wang, and Fabio Poiesi. Vocabulary-free 3d instance segmentation with vasion-language assistant, In 2025 International Conference on 3D Vision (3DV), pp. 1197- 1210. IEEE, 2025.
Phuc Nguyen, Tuan Due Ngo, Evangelos Kalogerakis, Chuang Gan, Anh Tran, Cuong Pham, and Khoi Nguyen. Open3dis: Open-vocabulary 3d instance segmentation with 2d mask guidance. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4018- 4028, 2024.
Songyou Peng, Kyle Genova, Chiyu Jiang, Andrea Tagliasacchi, Marc Pollefeys, Thomas Funkhouser, al. Openscene: 3d scene understanding with open vocabularies. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 815-824, 2023.
Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 652 660, 2017a.
Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++:: Deep hierarchical fea- ture learning on point sets in a metric space. Advances in neural information processing systems, 30, 2017b.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp- 9748-8763. PmLR, 2021.
David Rozenberszki, Or Litany, and Angela Dai. Language-grounded indoor 3d semantic segmen- tation in the wild. In European conference computer vision, pp. 121-141. Springer, 2022.
Jonas Schult, Francis Engelmann, Theodora Kontogianni, and Bastian Leibe. Dualcomvmesh-nett Joint geodesic and euclidean convolutions on 3d meshes. Proceedings of the IEEE/CVF con- ference computer vision and pattern recognition, pp. 8612 8622, 2020.
C. E. Shannon. A mathematical theory of communication. The Bell System Technical Journal, 27 (3):3794423, 1948. doi: 10.1002/j 7305. :b013333..
Boyi Sun, Yuhang Liu, Xingxia Wang, Bin Tian, Long Chen, and Fei-Yue Wang. 3d annotation- free learning by distilling 2d open-vocabulary segmentation models for autonomous driving. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 7078-7086, 2025.
Haoxiang Wang, Pavan Kumar Anasosalu Vasu, Fartash Faghri, Raviteja Vemulapalli, Mehrdad Farajtabar, Sachin Mchta, Mohammad Rastegari, Oncel Tuzel, and Hadi Pouransari. Sam-clip: Merging vision foundation models towards semantic and spatial understanding. In Proceedings of the IEEE/CV Conference Computer Vision and Pattern Recognition, pp. 3635-3647, 2024a.
Pengfei Wang, Yuxi Wang, Shuai Li, Zhaoxiang Zhang, Zhen Lei, and Lei Zhang. Open vocab- ulary 3d scene understanding via geometry guided self-distillation. In European Conference on Computer Vision pp. 460- Springer, 2024b.
12

---

Published as a conference paper at ICLR 2026
Yan Wang, Baoxiong Jia, Ziyu Zhu, and Siyuan Huang. Masked point-entity contrast for open- vocabulary 3d scene understanding. In Proceedings of the Computer Vision and Pattern Recogni- tion Conference, pp. 14125-14136, 2025.
Ziyi Wang, Yanbo Wang, Xumin Yu, Jie Zhou, and Jiwen Lu. Xmask3d: Cross-modal mask reason- ing for open vocabulary 3d semantic segmentation. Advances in Neural Information Processing Systems, 37:74922 74944, 2024c.
Wenxuan Wu, Zhongang Qi, and Li Fuxin. Pointconv: Deep convolutional networks on 3d point clouds. In Proceedings of the IEEE/CVF Conference computer vision and pattern recognition, pp. 9621-9630, 2019.
Xiaoyang Wu, Li Jiang, Peng-Shuai Wang, Zhijian Liu, Xihui Liu, Yu Qiao, Wanli Ouyang, Tong He, and Hengshuang Zhao. Point transformer v3: Simpler faster stronger. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4840-4851, 2024.
Xiaoyang Wu, Daniel DeTone, Duncan Frost, Tianwei Shen, Chris Xie, Nan Yang, Jakob Engel, Richard Newcombe, Hengshuang Zhao, and Julian Straub. Sonata: Self-supervised learning of reliable point representations. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 22193-22204, 2025.
Monika Wysoczańska, Oriane Siméoni, Michaël Ramamonjisoa, Andrei Bursuc, Tomasz Trzciński, and Patrick Pérez. Clip-dinoiser: Teaching clip few dino tricks for open-vocabulary semantic segmentation. In European Conference on Computer Vision, pp. 320-337. Springer, 2024.
Mutian Xu, Runyu Ding, Hengshuang Zhao, and Xiaojuan Qi. Paconv: Position adaptive convolu- tion with dynamic kernel assembling on point clouds. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 3173-3182, 2021.
Wei Xu, Chunsheng Shi, Sifan Tu, Xin Zhou, Dingkang Liang, and Xiang Bai. A unified framework for 3d scene understanding. Advances in Neural Information Processing Systems, 37:59468-
59490, 2024.
Jihan Yang, Runyu Ding, Weipeng Deng, Zhe Wang, and Xiaojuan Qi. Regionple: Regional point- language contrastive learning for open-world 3d scene understanding. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 19823-19832, 2024.
Junbo Zhang, Runpei Dong, and Kaisheng Ma. Clip-fo3d: Learning free open-world 3d scene representations from 2d dense clip. In Proceedings of the IEEF/CVF international conference on computer vision, pp. 2048-2059, 2023.
Shiqi Zhang, Sha Zhang, Jiajun Deng, Yedong Shen, Mingxiao Ma, and Yanyong Zhang. Pgov3d: Open- vocabulary 3d semantic segmentation with partial-to-glote curriculum. In Proceedings of the 33rd ACM International Conference on Multimedia, pp. 5070-5079, 2025.
Weiguang Zhao, Yuyao Yan, Chaolong Yang, Jianan Ye, Xi Yang, and Kaizhu Huang. Divide and conquer: 3d point cloud instance segmentation with point-wise binarization. Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 562-571, 2023.
Mingkai Zheng, Fei Wang, Shan You, Chen Qian, Changshui Zhang, Xiaogang Wang, and Chang Xu. Weakly supervised contrastive learning.
Proceedings of the IEEF/CVF International Conference on Computer Vision, pp. 10042-10051, 2021.
Xueyan Zou, Zi-Yi Dou, Jianwei Yang, Zhe Gan, Linjie Li, Chunyuan Li, Xiyang Dai, Harkirat Behl, Jianfeng Wang. Lu Yuan, et al. Generalized decoding for pixel, image, and language. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 15116- 15127, 2023.
13

---

Published as a conference paper at ICLR 2026
A EXPERIMENTAL SETUP AND IMPLEMENTATION DETAILS
This section provides the implementation details for experiments to ensure reproducibility. All 3D sparse operations are implemented using the Minkowski Engine (Choy et al., 2019). Our code and pre-trained models will be made publicly available upon publication.
A.1 NETWORK ARCHITECTURES
We detail the architectures for the student, teacher, and 2D semantic backhone networks used in our experiments.
Student Network (Øs): The student network, which we term the AffinityPredictor, is sparse 3D convolutional network that processes point-wise features. The input is sparse tensor where each voxel contains a 518-dimensional feature vector, formed by concatenating a 512-dim semantic feature from Mn with 6-dim geometric attribute vector (RGB and surface normal). The architecture comprises three stages:
1. An input stem that uses 3 x 3 3 sparse convolution, followed by batch normalization and ReLU, to project the 518-dim input features to a 512-dim hidden space. 2. A body composed of four sequential wwl.kkwssiiseBBlee convolutions with batch normalization, a residual connection, and a final ReLU activation. 3. A projection head that employs 1 x 1 x 1 sparse convolution map the 512-dim features to the final 128-dim embedding.
modules. Each block contains two 3
sparse
The complete architecture is detailed in Table 5.
Table 5: Student Network Architecture (os). The network operates on sparse 3D tensors. The feature dimension maintained at 512 throughout the body.
Stage
Operator
Kernel Size
Stride
Channels (In - Out)
Input
ME, www.kwwkki.onooolttttt..... ion, ME, Bat ME, ReLL
ga
518 - 512
x MinkowskLRResBlec.
512 - 512
MiinkuwskiiCnnee Lon, BatchNorm,
33
512 512
Body
MP, Minkowwkk Canvolut ion, ME, BatchNorm
3
512 512
+ (Identity Shortcut)
ReLU
Head
WgMMinkkowskiiioonooltttiio
128
Teacher Network (ØT): The teacher network is a frozen, pre-trained Sonata model (Wu et al., 2025), 3D self-supervised model built upon the Point Transformer V3 (PTv3) architecture. We employ this model for its high-quality, 3D-native representations, which serve as the source of sta- ble geometric priors for distillation. The efficacy of these features is demonstrated on the Scan- Net benchmark, where they achieve 72.5% mloU with linear probing, substantially outperforming 2D-lifted features from models like DINOv2 (63.1% mloU). The original publication further sub- stantiates the robustness of Sonata's features through qualitative analyses that show clear semantic clustering and strong spatial reasoning. We use the officially released checkpoint and refer readers to the original work for complete architectural details.
2D Semantic Backbone (V2p): The 2D semantic backbone provides initial semantic priors us- ing frozen, pre-trained X-Decoder with a DaViT-L backbone (Zou et al., 2023), for which we use the officially released weights. X-Decoder is general-purpose model trained on diverse tasks (e.g., segmentation, image-text retrieval) to learn a unified visual-linguistic embedding space. We hypothesize that features learned via this multi-modal multi-tast task objective provide richer semantic information for 3D scene understanding than those from networks trained solely on a single task like segmentation.
14

---

Published as a conference paper at ICLR 2026
A.2 DATASET DETAILS AND SUBSET SELECTION
For all experiments, we adhere the official training, validation, and testing splits for the Scan- NetV2 and Matterport3D datasets to ensure fair comparison with prior work.
Algorithm Principled Scene Subset Selection
Input: Full set of regions R. desired subset size K Kaubset, number of clusters Kelusters, richness weight y
Output: Selected optimal subset regions Reelected.
Phase 1: Metric Calculation
1:
Let List +- |]
for each region E do Extract semantic labels from r.
4: Ne(r) <- Count of unique lahels in
b. Semantic Richness
5: p(c) <- Proportion of each lahel i 1.
6:
H.(r)
Σ. p(c) log, p(c).
Semantic Complexity (Entropy)
His((r) < Full histogram of labels I.
8:
Append (r, Nr(),, Hr(r), Hist(r)) to StatsList.
9: end for
Phase Filtering
10: Ne Mediann (s.Ne for E StatsLis()).
11: He + Median (s. for s E StatsList)).
FilteredList + for StatsList s.Ne 2 No and s.H. 2 He].
Phase 3: Clustering
13: F <- Hist for E FilteredTist].
Create feature matrix from histograms
KMeans(F. (F, Kotusters) for +- to length(Filllerede (iinetdddl do (illeredliss(i) & Cluster Labels(] 17: end for
14: ClusterLabels
16:
Phase Scoring
Nomnrrm <- MinMaxScaler((e (s.. Ns.Ne for s FilteredList)).
19: Hemorm +- MinMaxScaler([ H, for s FilteredLåst)..
20: for 4- to length(Fittereeddas))) do 21:
22: end FilteredList(i]
Hanorr [8] 7 Ne
Phase 5: Stratified Selection
23: Let Reelected +-1
subsset/ Ceiieet//cluseeee Kelusters] 24: Maase |
25: nrem +- Kaubuct (mut Keinsters)
26: for 100 Kelustera
t 27: Cluster; <- {s for E FilleredList if s.cluster = j).
do
28: Sort Cluster, in descending order by s.score.
29: Nsclect Rbase + (1 if < Nrem else 0).
30: Add top nsetect regions from sorted Clustery to Roclected.
31: end for
32:
return Reelected.
To substantiate our claims of data efficiency, we select compact training subset using the principled, automated pipeline detailed Algorithm This method is configured with key hyperparameters. To align with the categorical diversity of each dataset, the number of clusters K is set based on the variety of real-world spaces documented in the original papers: we use K = 20 for ScanNetV2 and K 12 for Matterport3D. The michess-complees
score weight is held constant 7 0.5. This process yields final subset 20 scenes and 20 regions, respectively. The complete lists the selected identifiers provided in Table 6.
15

---

Published as a conference paper at ICLR 2026
Table 6: Final scene and region IDs constituting our compact training subsets for ScanNetV2 and
Matterport3D.
ScanNetV2 Selected Scenes
Matterport3D Selected Regions
acene0126 00
VzgfbhrpDEA region7
scene0640 02
cV4RVežvušT region
acene0247_C1
739xcd9rjKW5rreeioon12
scene0604_00
ULsKac@VVFJR.reeioonnI
scene0315_00
ubbo@@7RLL61X_ree11on333
scene0604_02
VYXNIzi1PKCB.reegioon377
scene01C2.C1
D7N2EECCX455__eegooon1
scene0137.0
159xx97/]WW52eee1onn16
scene0588 03
sceme0547_01
mnJbRahghhzi..eeiionn
sceme0451.02
VI2agDo3317E_reeiionn
acene0341.00
VL2zgDo33175.reggionn
scene0106_00
VzoffbhrpDDEa.regiion2
scene0592.00
B6GyNeegPKKss.eggi.nn8
scene0692.00
D7N2EECX445__eegionn]
scene0393.00
D7N2EECXX46jrregioon38
sceme0340_00
2m8k4RJN13HMrree11o011
scene0220_00
DScIXDM8gDDG.eegionn1
scene0394
Frraa06n20ommzzeioon.
scene0692.01
82sE5b5ptXE.regiion
A.3 COMPREHENSIVE HYPERPARAMETER TABLE
To ensure full reproducibility of our experimental results, we provide comprehensive list of all hy- perparameters used for model training, distillation, and inference. Our final configuration is detailed in Table 7.
B ADDITIONAL QUANTITATIVE RESULTS
This section provides a detailed breakdown of our method's performance to supplement the sum- mary results in the main paper and offer concrete evidence for our core technical contributions.
B. PER-CLASS SEGMENTATION RESULTS
The per-class loU scores in Tables and 9 offer a granular validation of our central hypothesis: that explicitly purifying the geometric structure of projected 2D features is critical for robust 3D> segmentation. The baseline, which relies directly on multi-view aggregated features from the X- Decoder VLM, suffers from the geometric inconsistencies inherent view projection, leading to fragmented and noisy object representations. Our method, GeoPurify, systematically addresses this deficiency.
On ScanNetV2 (Table 8), GeoPurify improves mloU by 4.9%. The most substantial gains are ob- served in categories where object identity is strongly tied geometric structure rather than just texture color. For instance, classes with intricate parts and thin structures, such chair (+8.5%), sofa (+7.1%) 1,), and desk (+5.6%) see dramatic improvements. This a direct result of our Geometry- Guided Pooling mechanism. The bascline often fractures the features of chair's legs and backrest, mixing them with features from the background. work learns to establish high geometric affinity among all points belonging to the chair's structure. The subsequent pooling step then uses this affinity graph to enforce semantic consistency, effectively "denoising" the initial VLM features and producing coherent object representation.
contrast, our contrastively trained student net-
similar pattern emerges on the more complex Matterport3D benchmark (Table 9), where our method attains 2.7 % higher mloU. The significant improvements for classes like toilet (+6.8%)
16

---

Published as a conference paper at ICLR 2026
Table 7: Comprehensive list of hyperparameters. All settings kept consistent across datasets
unless otherwise specified.
Hyperparameter
Value
Training Parameters
Optimizer
AdamW
Base Learning Rate (Irssse
le-4
Learning Rate Scaling
Input layer: 0.1 Ifbase
Middle layers: 1.0 X Inase
Output layer: 5.0 X Irease
LR Scheduler
Cosine Annealing with Linear Warmup
Warmup Epochs
Warmup Start Factor
1e-6
Cosine Annealing Min LR (7min)
Ic-7 (10-3
3 x Irbase)
Weight Decay
le-5
Total Epochs
50
Contrastive Distillation
InfoNCE Temperature (r)
0.07
Anchor Points (per scene) nacro-nega (per anchor)
4096
48
Micro-negatives (per anchor)
Inference & Pooling
Pooling Iterations (7)
Affinity Sharpening Factor (a)
0.05 (1/20.0)
Nearest Neighbors for Affinity (K)
96
Voxel Size
0.02
and curtain (+5.9%) further validate our approach. These objects often have uniform textures where 2D appearance can ambiguous; their defining characteristic is their 3D shape. GeoPurify excels here because our geometry-centric student network provides the precise structural cues needed to group points correctly, overcoming the semantic ambiguity of the initial 2D features.
Conversely, the results also highlight the current limitations of our local pooling strategy. Classes defined by extremely thin geometry and strong co-planar relationship with larger surface, such as picture and shower curtain, show minimal gains even slight degradation. In these scenarios, the local neighborhood queried by our pooling mechanism is dominated by the larger surface (e.g.. wall). Consequently, the powerful semantic features of the wall can "leak" into the picture's features during the affinity-based aggregation, overwhelming its distinct identity. This indicates that while method robustly handles complex object-level geometry, resolving features for extremely fine- grained, surface-level details remains a challenge for future work.
Table 8: Per-class IoU (%) on the ScanNetV2 validation set. mloU: Baseline 50.2, GeoPurify (Ours) 55.1 (+4.9). mAce: Baseline 68.1, 1, GeoPurify (Ours) 72.5 (+4.4).
dbai
50
uble
windoww
Cunmic
desk
curtain
tridge
ShOwAr
solles
bobbeb
303
GenPurity (Oun)
65.7
71.6
51.8
39.0
30.9
36.4
54,8
49.6
+3.0
+5.0
+1.5
+4.1
+3.5
+7.1
+7.8
+1.4
+5.1
+3.3
+2.4
+5.6
+1.2
+7.2
+5.0
+8.9
+4.8
+5.1
Table 9: Per-class IoU (%) on the Matterport3D test set. mloU: Baseline 37.5, GeoPurify (Ours)
40.2 (+2.7). mAce: Baseline 59.8, GeoPurify (Ours) 62.4 (+2.6).
Method
cahinos
bed
s0h
Window
pkaure
deck
cumain
frige
bahuab
Rescline
63.
11.7
37.7
67.1
33.2
313
360
41.8
450
11.5
63
16.6
15.0
41.4
25.
638
41.5 26
13.5
140
GesPurify (Oues)
63.5
73.3
39,1
71.3
40.6
84.7
18.2
46.7
42.1
11.1
011
16.0
19.8
49.3
27.7
7.6
48.3
27.3
39.0
76.4
&
+1.7
+1.6
+1.4
+42
+5.4
+3.4
+2.2
+3.9
+33
-04
-0.
0.6
+3.0
+5.9
+2.7
40.8
+6.8
+1.
+3.7
+2.4
17

---

Published as a conference paper at ICLR 2026
B.2
GENERALIZATION
As demonstrated in the main text, our decoupled geometric prior provides robust generalization across different 3D domains. This section presents more granular analysis to further probe the limits of this robustness, subjecting the model to a simultaneous shift in both geometric distribution and semantic vocabulary size. To this end, evaluate model trained ScanNetV2 increas- ingly larger subsets of the Matterport3D dataset, defined by the K most common categories for K {40, 80, 160).
The results, shown in Table 10, reveal that the performance advantage of GeoPurify over competing methods magnifies the vocabulary expands. This trend starkly illustrates the failure mode of models with entangled geo-semantic representations. When confronted with a large set of unfamiliar classes, their geometric reasoning falters, of the source domain. This leads to rapid degradation in performance as vocabulary size increases.
implicitly conditioned on the semantic distribution
In contrast, the superior scaling of our method validates the benefit of a decoupled architecture. Because GeoPurify's s student network learns purely structural, class-agnostic prior, its ability to enforce geometric coherence is not compromised by the novelty diversity of the semantic features. It continues to function as powerful regularizer, denoising the initial 2D features and preserving structural integrity. This confirms that our method's s robustness a fundamental property that scales effectively to more challenging and diverse open-world scenarios.
Table 10: Comparison cross-dataset generalization against state-of-the-art methods. All models are trained on ScanNetV2 and tested zero-shot on Matterport3D. We evaluate on the original 21- class benchmark and on subsets containing the K most common categories from the NYUv2 label set, where K E {40, 80, 160).
Method
Matterport21 mAce
Matterport40 mAce
Matterport80 mIoU
Matterport160
mIoU
mIoU
mAce
mloU
mAcc
OpenScene (Peng et al., 2023)
36.0
48.0
21.1
27.5
10.8
13.9
6.0
8.1
CUA-O3D (Li et al., 2025)
37.4
49.2
23.3
30.2
12.2
16.3
6.1
8.4
GGSD (Wang et al., 2024b)
40.1
54.4
22.8
31.6
11.9
16.2
6.3
9.6
GeoPurify (Ours)
40.5
62.7
33.3
48.3
19.1
32.8
12.3
21.6
C SUBSET SELECTION STRATEGY
To ensure that the efficacy of GeoPurify is not an artifact of the specific data sampling method, we evaluate its robustness against two fully annotation-free alternatives: random uniform selection and unsupervised clustering based on spatial and colorimetric features. As reported in Table 11, the model trained on randomly selected scenes achieves an mloU of 54.6%, representing a negligible deviation from the heuristic baseline. Similarly, the unsupervised clustering strategy yields 54.9% mloU. These findings confirm that the observed performance gains are intrinsic to the proposed ge- ometric purification mechanism and do not rely specific high-quality samples global statistical guidance.
Table 11: Robustness analysis subset selection strategies on ScanNetV2 (~1.5% data). The comparison demonstrates that GeoPurify maintains consistent performance across varying selection methodologies.
Selection Strategy
mloU
mAcc
Heuristic (Ours)
55.1
72.5
Random
54.6
72.2
Coordinate and Color Clustering
54.9
72.3
18

---

Published as a conference paper at ICLR 2026
D EXTENDED QUALITATIVE ANALYSIS
This section provides a detailed qualitative analysis of method on the ScanNetV2 and Matter- port3D datasets to offer comprehensive understanding of its capabilities and limitations.
D.1 VISUALIZATION OF INTERMEDIATE MECHANISMS
Figure provides a conceptual illustration of our Crometry-Guudeee Pooling module, which is math- ematically detailed in the main text. This mechanism is designed to rectify the geometric inconsis- tencies present the imitial semantic features that are aggregated from multi-view 2D images.
The process begins with the trained student network, Øs. which generates distinct geometric em- bedding for each point in the scene. From these embeddings, we construct a sparse affinity matrix that encodes structural relationships within local neighborhoods. The strength of the affinity be- tween two neighboring points is derived from the sharpened cosine similarity of their embeddings, concentrating the connection weights onto the most structurally similar points-for instance, those lying on the same continuous surface (visualized as green lines). The purification itself is an it- erative refinement, where the semantic features are updated each step by being multiplied with this affinity matrix. This operation functions as geometry-award filter, effectively propagating and averaging semantic information across structurally coherent regions. As depicted, this process de- noises the initial fragmented features, yielding final representation that is both semantically rich and geometrically sound.
High-Aliiily Low-Aflinit
Query Point
Geodelry-Guuded Pooling C+
Initial 3D Features
Refined 3D Features
Figure 3: The Geometry-Guided Pooling Mechanism. Left: Initial point cloud features aggre- gated from 2D views exhibit significant fragmentation and geometric inconsistency. Center: Our pooling mechanism operates locally. For given query point, the learned affinity matrix identifies nearby points the same surface having high affinity (green lines) while assigning low affinity to points on other surfaces (gray lines). Right: By iteratively averaging features among high-affinity neighbors, the process denoises the initial features, producing final output that is both semantically rich and geometrically coherent.
D.2 QUALITATIVE RESULTS
We first provide visual analysis feature coherence before and after our proposed purification. As illustrated in Figure 4, initial features aggregated from 2D views exhibit significant fragmentation. Visualized via k-means clustering, these features yield scattered, incoherent segments that do not align with the underlying object geometry. In contrast, after applying GeoPurify's (reometry-Guidee Pooling, the refined features produce contiguous clusters that precisely match distinct surfaces, such walls, floors, and furniture. This result demonstrates our method's effectiveness in instilling structural awareness into the 3D feature space.
Next, we evaluate the impact of this enhanced feature quality on open-vocabulary semantic seg- mentation. Figure presents a side-by-sid comparison in challenging scenes from the ScanNetV2 validation and Matterport3D test sets. The baseline, which uses unpurified features, generates noisy predictions with inaccurate boundaries and fragmented object detections. GeoPurify consistently mitigates these artifacts, producing segmentation maps that are substantially more coherent and pre-
19

---

Published as a conference paper at ICLR 2026
Figure 4: Visualization of Feature Coherence. Point clouds colored by applying k-means clustering to their features. Top Row (Initial Features); Features aggregated directly from 2D views lead to fragmented and geometrically inconssstent clusters. Bottom Row (Purified Features): Our Geometry-Guided Guided Pooling produces smooth, coherent features that align with the underlying 3D structure.
Input
Ground Truth
(hogroggnted features)
Baseine
GeaPuris
Figure 5: Qualitative Comparison on ScanNetV2 and Matterport3D. Visual results on chal- lenging indoor scenes. From top to bottom: Input RGB point cloud, Ground Truth segmentation, Baseline predictions (from aggregated 2D features), and GeoPurify predictions. Our method generates significantly cleaner segmentation maps with sharper boundaries and more complete ob- ject segments compared to the baseline.
cise. Our method successfully delineates sharp boundaries and captures object instances with greater completeness, yielding results that more closely approximate the ground truth.
D.3 FAILURE CASE ANALYSIS
To provide a transparent assessment, we analyze the primary failure modes our model, as shown in Figure 6. We identify principal categories of errors:
20

---

Published as a conference paper at ICLR 2026
Baseline
Input RGB
Ground Truth
(Aggregated 20 features)
Geolarity
Oversmenthing
Figure 6: Illustration of typical failure modes. From left right: challenges with the presence of weroshng artifacts at object boundaries, and inherited semantic enrors from the initial 2D feature backbone.
Over-smoothing Artifacts: In some instances, the iterative pooling process can subtly blur the boundaries between distinct but closely adjacent objects, resulting in minor local inaccuracies.
Semantic Errors from 2D Backbone: The performance GeoPurify dependent on the quality of the initial 2D features. When the 2D backbone generates fundamental semantic error, our geometric purification module is often unable correct the misclassification.
These limitations highlight key challenges and suggest promising directions for future research, in- cluding the development of more robust feature fusion techniques and error correction mechanisms.
E THE USE OF LARGE LANGUAGE MODELS
A large language model (LLM) was utilized writing assistant in the preparation of this manuscript. The scope of its use was strictly limited to article polishing, including improving gram- mar, enhancing clarity, and ensuring conciseness. All core scientific contributions, such research ideation, methodology development, and experimental analysis, were conceived and conducted by the human authors. The authors have reviewed all LLM-generated text and take full responsibility for the scientific accuracy and integrity of the final content.
APPENDIX REFERENCES
Christopher Choy, Jun YunYoun Gwak, and Silvio Savarese. 4d spatio-temporal convnets: Minkowski convolutional neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 3075-3084, 2019.
Jinlong Li, Cristiano Saltori, Fabio Poiesi, and Nicu Sebe. Cross-modal and uncertainty- aware agglomeration for open-vocabulary 3d scene understanding. In Proceedings of the Computer Vision and Puttern Recognition Conference, pp. 19390-19400, 2025.
Songyou Peng, Kyle Genova, Chiyu Jiang, Andrea Tagliasacchi, Mare Pollefeys, Thomas Funkhouser, et al. Openscene: 3d scene understanding with open vocabularies. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 815-824, 2023.
21

---

Published as a conference paper at ICLR 2026
Pengfei Wang, Yuxi Wang, Shuai Li, Zhaoxiang Zhang, Zhen Lei, and Lei Zhang. Open vocab- ulary 3d scene understanding via geometry guided self-distillation. In European Conference on Computer Vision, pp. 442-460 Springer, 2024.
Xiaoyang Wu, Daniel DeTone, Duncan Frost, Tianwei Shen, Chris Xie, Nan Yang, Jakob Engel, Richard Newcombe, Hengshuang Zhao, and Julian Straub. Sonata: Self-supervised learning of reliable point representations. Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 22193-22204 2025.
Xueyan Zou, Zi-Yi Dou, Jianwei Yang, Zhe Gan, Linjie Li, Chunyuan Li, Xiyang Dai, Harkirat Behl, Jianfeng Wang, Yuan, et Generalized decoding for pixel, image, and language. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 15116- 15127, 2023.
22


Reviews:
[{"review_id": "jwjmxqofCn", "reviewer": "Reviewer_La46", "summary": "This paper presents GeoPurify, a geometric distillation framework for open-vocabulary 3D semantic segmentation. By introducing geometry-guided contrastive distillation and affinity-based semantic purification, the method aims to reduce projection noise from 2D VLM features and improve 3D semantic consistency. The approach demonstrates strong results on datasets and requires only 1.5 % of the trainning data, showing satisfactory annotation efficiency.", "questions": "1. Could the authors provide a quantitative comparison with recent VLM-based understanding pipelines?\n\n2. How robust is GeoPurify if the subset is selected randomly instead of using global statistics?\n\n3. How does the method perform on outdoor datasets such as nuScenes or SemanticKITTI?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 1, "strengths": "1. The idea of leveraging geometric priors to purify VLM features is meaningful and addresses a real weakness of projection-based pipelines.\n\n2. The method achieves competitive performance with significantly reduced 3D training data, which is practically valuable for indoor scene understanding.", "weaknesses": "1. The claimed paradigm shift from *“Segmentation-and-Matching”* to *“Segmentation-as-Understanding”* is not entirely novel, as recent works such as OV3D[1] and PGOV3D[2] already adopt VLM-based understanding pipelines. The conceptual contribution could be more precisely positioned, and comparisons with these recent methods are somewhat limited.\n\n[1] Jiang, Li, Shaoshuai Shi, and Bernt Schiele. \"Open-vocabulary 3d semantic segmentation with foundation models\" in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024.\n\n[2] Zhang, Shiqi, et al. \"PGOV3D: Open-Vocabulary 3D Semantic Segmentation with Partial-to-Global Curriculum\" in Proceedings of ACM Multimedia, 2025.\n\n2. The current subset selection depends on global dataset statistics, which contradicts the “fully annotation-free” motivation. It remains unclear whether the reported gains are due to the method itself or a bias toward selecting higher-quality samples.\n\n3. The method is evaluated only on indoor datasets (e.g., ScanNet). Since the approach relies heavily on geometric consistency, its generalization to outdoor datasets such as nuScenes remains unclear.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "eW6sQwGwxO", "reviewer": "Reviewer_tyeq", "summary": "The paper introduces GeoPurify, a method for open-vocabulary 3D semantic segmentation that enhances the 3D consistency of features extracted from 2D vision-language foundation models. The approach first extracts 2D features using X-Decoder and merges them in 3D via a student affinity network trained to learn geometric relationships by distilling knowledge from the 3D foundation model Sonata (Wu et al.). Training is performed on only a small subset of ScanNetV2 (1.6%) and Matterport3D (1.3%), demonstrating data efficiency. Evaluation on ScanNetV2, Matterport3D, and ScanNet200 shows that GeoPurify outperforms training-based baselines trained on similar subsets, and performs competitively-though slightly worse-than full-data and zero-shot baselines.", "questions": "1. Would it be possible to elaborate on why in Table 1 GeoPurify does not improve much over the zero shot baselines?\n2. I believe the method can be interesting even if trained on more data. Therefore, it would be interesting to evaluate its performances when trained on the whole datasets and see whether in that way it can improve over the other baselines.\n3. Is the point cloud required as input at inference time? From the text it seems to be the case (line 155), but from Figure 2 it seems to be needed only at training time.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. **Effective use of Sonata for geometric distillation.** The paper presents an interesting idea of enhancing the 3D consistency of 2D foundation model features by distilling geometric relationships from a strong 3D foundation model (Sonata). This cross-modal distillation strategy is conceptually sound and well motivated.\n2. **Clear and well-structured presentation.** The paper is generally clearly written and easy to follow.", "weaknesses": "1. **Limited performance gains.** Although the method is conceptually interesting, it does not clearly outperform strong zero-shot baselines. This raises uncertainty about the practical benefit of the proposed geometric distillation mechanism.\n2. **Restricted evaluation scope.** The method is only evaluated in a low-data regime. Assessing performance when trained on larger portions of the dataset would provide a clearer picture of its scalability and potential advantages over existing approaches", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "aJiwnQU0bD", "reviewer": "Reviewer_MQX3", "summary": "This paper presents GeoPurify, a data-efficient geometric distillation framework for open-vocabulary 3D segmentation. The work identifies a fundamental disconnect between semantic richness and geometric coherence in existing methods and proposes a shift from the \"segmentation and matching\" paradigm to \"segmentation as understanding.\" The core idea is to leverage a student affinity network to distill latent geometric structure from noisy 2D VLM-generated 3D features under the guidance of a self-supervised 3D teacher model. A geometry-guided pooling mechanism is further introduced to refine features during inference. The method is evaluated on multiple benchmarks and shows competitive or state-of-the-art performance using only ~1.5% of the training data, demonstrating strong data efficiency and generalization ability.", "questions": "This paper presents a well-designed framework that effectively integrates contrastive distillation and geometric pooling, but it fails to clearly demonstrate how its core idea of learning geometric affinity structures provides advantages beyond existing methods.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper clearly articulates the limitations of existing approaches in reconciling 2D semantics with 3D geometry and proposes a principled shift in perspective toward \"segmentation as understanding.\" The idea of recovering latent geometric information from noisy 2D features, rather than learning 3D geometry from scratch, is both novel and well-motivated.\n\n2. The paper provides extensive evaluations across multiple datasets. The comparison with a strong baseline under the same low-data regime is particularly compelling.", "weaknesses": "1. Some figures contain low-resolution text and should be improved. The authors should consider replacing these with higher-resolution images or vector graphics to improve readability.\n\n2. Heavy reliance on pre-trained models; the contribution breakdown is unclear. The performance gains of GeoPurify are built upon powerful pre-trained models. While the framework itself is novel, it remains unclear how much of the improvement comes from the proposed distillation and pooling mechanisms versus the strength of the pre-trained backbones.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kO1fHroLGY", "reviewer": "Reviewer_1k2F", "summary": "This paper proposes an open-vocabulary 3D segmentation framework, GeoPurify, which purifies noisy semantic features using 3D geometric guidance. The approach introduces a lightweight Student Affinity Network that learns pure geometric relationships via a Geometric Contrastive Distillation scheme, without requiring any 3D semantic labels. The method demonstrates notable data efficiency, as using only ~1.5% of the training scenes achieves comparable performance to state-of-the-art methods trained on the full dataset.", "questions": "Please refer to details in the above section.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The method achieves SOTA performance using ~1.5% of unlabeled 3D scan data, significantly reducing reliance on costly 3D annotations, offering a practical and promising solution for 3D understanding in data-scarce scenarios.\n2. This paper learns a pure geometric prior to regularize and enhance semantic features, which addresses the key issue of geometric fragmentation resulting from the 2D-to-3D projection.", "weaknesses": "1. The choice of 20 scenes lacks empirical justification. An ablation study examining performance trends across different subset sizes (e.g., 10, 30, or 50 scenes) is missing, making it difficult to assess the optimality of the selected size or the method's sensitivity to training data quantity.  \n2. Would there be more analysis on the effectiveness of data efficiency? Table 1 presents the severe degradation of CUA-O3D under a ~1.5% subset. Why does the proposed method not show such a degradation? \n3. Lack of discussions with related methods. Recent state-of-the-art methods, such as Open3DIS [1] and UniSeg3D [2], have reported results on ScanNet200, which are not discussed in this paper. Are the experimental settings different?\n\n[1] Open3DIS: Open-Vocabulary 3D Instance Segmentation with 2D Mask Guidance. CVPR 24.\n\n[2] A Unified Framework for 3D Scene Understanding. NeurIPS 24.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)