Tackling View-Dependent Semantics in 3D Language Gaussian Splatting
Authors: Jiazhong Cen
Xudong Zhou
Jiemin Fang
Changsong Wen
Lingxi Xie
XIAOPENG ZHANG
Wei Shen
Qi Tian
Venue: ICML 2025 poster (2025)
Keywords: 3D Gaussian splatting; open-vocabulary scene understanding; 3D scene understanding
Primary area: applications->computer_vision
TL;DR: This paper presents LaGa, a language-driven open-vocabulary 3D scene understanding method built upon 3D Gaussian splatting, designed to effectively handle the view dependency of 3D semantics.

Abstract: Recent advancements in 3D Gaussian Splatting (3D-GS) enable high-quality 3D scene reconstruction from RGB images. Many studies extend this paradigm for language-driven open-vocabulary scene understanding. However, most of them simply project 2D semantic features onto 3D Gaussians and overlook a fundamental gap between 2D and 3D understanding: a 3D object may exhibit various semantics from different viewpoints—a phenomenon we term **view-dependent semantics**. To address this challenge, we propose **LaGa** (**La**nguage **Ga**ussians), which establishes cross-view semantic connections by decomposing the 3D scene into objects. Then, it constructs view-aggregated semantic representations by clustering semantic descriptors and reweighting them based on multi-view semantics. Extensive experiments demonstrate that LaGa effectively captures key information from view-dependent semantics, enabling a more comprehensive understanding of 3D scenes. Notably, under the same settings, LaGa achieves a significant improvement of **+18.7\% mIoU** over the previous SOTA on the LERF-OVS dataset. Our code is available at: https://github.com/https://github.com/SJTU-DeepVisionLab/LaGa.

Tackling View-Dependent Semantics in 3D Language Gaussian Splatting
Jiazhong Cen
Xudong Zhou'
Jiemin Fang²
Changsong Wen Lingxi Xie? Xiaopeng Zhang? 2
Wei Shen
Qi Tian
Abstract
Recent advancements 3D Gaussian Splatting (3D-GS) enable high-quality 3D scene reconstruc- tion from RGB images. Many studies extend this paradigm for language-driven open-vocabulary scene understanding. However, most of them simply project 2D semantic features onto 3D Gaussians and overlook a fundamental gap between 2D and 3D understanding: a 3D object may exhibit various semantics from different viewpoints-2 phenomenon we term view- dependent semantics. To address this challenge, we propose LaGa (Language Gaussians), which establishes cross-view semantic connections by decomposing the 3D scene into objects. Then, it constructs view-aggregated semantic representations by clustering semantic descriptors and reweighting them based on multi-view semantics. Extensive experiments demonstrate that LaGa effectively captures key information from view-dependent semantics, enabling more comprehensive understanding of 3D scenes. Notably, under the same settings, LaGa achieves significant improvement of +18.7% % mloU over the previous SOTA on the LERF-OVS dataset. Our code is available https: //github. SJTU- SCTUUDDeeVVisiontabbiie
(6) Pipeline Existing 2D Methods
Rosults (Lang
Viewpoint (Necessary)
Text Prompt
Procept: "Tyy Blephan!
CLIP
View
Renderol Fratue Map
3D Cunssian
Embodided
Segmentation Mark
View
(b) Pipeline Direct 3D Understanding
Pronge
Tleyhoor"
Noloy
Text Prompt
Viewpoint (Optional)
CLIP
View
Grussiazs Segmennation Resalt
Embeeded Gaussions
View
Figure I. Pipeline comparison of existing 2D methods (a) and the direct 3D scene understanding paradigm (b). While 2D methods excel in pixel-wise understanding via rendered feature maps, they fail unexpectedly when 3D Gaussians are directly retrieved by matching learned 3D features with CLIP text embeddings.
vocabulary scene understanding. To this end, recent stud- ies (Qin et al., 2024; Shi et al., 2024; Bhalgat et al., 2024; Peng et al., 2025; Cheng al., 2024; Qu al., 2024) extend 3D-GS by lifting multi-view 2D semantic features extracted from vision-language models like CLIP (Radford al., 2021) into the 3D feature representation. As shown in Fig- ure ((a), these methods heavily rely on the differentiable rasterization mechanism of 3D-GS. During training, they optimize the 3D features by aligning rendered feature maps with 2D semantic features at corresponding viewpoints. At inference, they continue rendering the learned 3D semantic features into 2D feature maps, using them for pixel-wise semantic understanding with cross-view consistency.
1. Introduction
3D Gaussian Splatting (3D-GS) (Kerbl et al., 2023), a recent breakthrough in radiance fields, offers superior rendering efficiency and quality. Its unstructured 3D point-cloud representation makes a promising foundation for open-
'Work done during internship at Huawei 'MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University "Huawei Technologies Co., Ltd.. Correspondence to: Wei Shen <wei.shen @ww.ee.sssuuu.......... Jiemin Fang <jamin- fong @ Long@gmai.ccom gmmil
However, when applying their learned 3D features to di- rect 3D perception (Wu et al., 2024b; Lee et al., 2025), as shown Figure 1(b), their effectiveness degrades signifi- cantly, limiting their applicability to downstream tasks such
Proceedings of the ndnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Tackling View-Dependent Semantics in 3D Language Gaussian Splatting
critical information across viewpoints.
PASSPORTTS
Despite its simplicity, LaGa significantly bridges the per- formance gap between 3D and 2D perception on existing benchmarks, achieving +18.7 % mloU over previous 3D methods and surpassing state-of-the-ard 2D results by +8.8% % mloU. Our approach offers a new perspective for advancing 3D semantic understanding within the 3D-GS framework.
PASPPURT
2. Related Work
2.1. Radiance Fields and 3D Gaussian Splatting
Figure 2. An illustration of view-dependent 3D semantics. The passport exhibits different semantics from different viewpoints.
Neural Radiance Fields (NeRF) (Mildenhall et al., 2020) pi- oneer using differentiable rendering for 3D scene reconstruc- tion from multi-view images. Numerous works enhance its rendering quality (Zhang et al., 2020; Martin-Brualla et al., 2021; Barron et al., 2021; 2022: 2023) and efficiency (Sun et al., 2022; Chen al., 2022; Lindell et al., 2021; Hedman et al., 2024; Reiser et al., 2021; Müller et al., 2022; Wizad- wongsa et al., 2021; Neff et al., 2021; Fridovich-Keil et al., 2022). Recently, 3D Gaussian Splatting (3D-GS) (Kerbl et al., 2023) uses explicit 3D Gaussians as 3D representation and differentiable rasterization for real-time, high-quality rendering. It inspires applications in 3D generation (Yi et al., 2024; Yang et al., 2024a; Tang al., 2025), 3D scene edit- ing (Chen et al., 2024; Wang et al., 2024), and 4D scene reconstruction (Yang et al., 2024b; Wu et al., 2024a; Duan et al., 2024; Mihajlovic et al., 2025). We build upon 3D-GS to achieve open-vocabulary 3D scene understanding.
as 3D object editing, Al-driven interaction, and precise 3D localization. We identify a fundamental issue behind this limitation: the view-dependency of 3D semantics. As illus- trated in Figure 2, passport viewed from the front clearly reveals its title, while from the back or side, it becomes unrecognizable. This phenomenon highlights a fundamental gap between 2D and 3D understanding. Simply projecting 2D semantics onto 3D Gaussians results in incomplete or inaccurate semantic assignments, as each Gaussian inherits semantics visible only from specific viewpoints. Specifi- cally, this leads to false positives (noisy Gaussians) and false negatives (incomplete results), as depicted in Figure 1 (bb).
To quantify this issue, we conduct two analyses. First, a semantic similarity distribution analysis shows that multi- view semantic features of the same object often exhibit lower intra-object similarities than inter-object similarities, provid- ing direct evidence of view-dependent semantics. Second, semantic retrieval integrity analysis finds that about 50% % of 2D semantic features fail to retrieve their corresponding 3D objects completely, further validating its negative affect.
2.2. 3D Scene Understanding in Radiance Fields
Semantic-Agnostie Scene Understanding. The growing popularity of radiance fields leads to research in interactive 3D segmentation and scene decomposition. NVOS (Ren et al., 2022) introduces the first interactive method for ob- ject selection in NeRFs. N3F (Tschernezki et al., 2022), DFF (Kobayashi et al., 2022), and ISRF (Goel et al., 2023) lift features from 2D self-supervised models (Caron et al., 2021) into 3D via learned feature fields. NeRF-SOS (Fan et al., 2023) and ContrastiveLift (Bhalgat et al., 2023) dis- till 2D feature similarities into 3D. More recently, with the advent of the Segment Anything Model (SAM) (Kirillov et al., 2023), studies such as Garfield (Kim et al., 2024), OmniSeg3D (Ying et al., 2024), SA3D (Cen al., 2023), GaGa (Lyu et al., 2024), and SAGA (Cen et al., 2025a) uses SAM- segmentation masks for scene decomposi- tion. Our approach builds upon this framework by using SAM-extracted masks to construct cross-view semantic con- nections and decompose the 3D scene into objects.
To address this challenge, we propose LaGa, a simple yet effective method for open-vocabulary 3D scene understand- ing with 3D-GS. LaGa first performs 3D scene decompo- sition by grouping multi-view 2D masks into coherent 3D objects, establishing cross-view semantic connections to ex- plicitly capture view-dependent 3D semantics. Then, LaGa constructs view-aggregated semantic representations by ex- tracting a representative set of semantic descriptors for each 3D object via adaptive clustering. To enhance robustness, LaGa assigns weights to descriptors based on two factors: (1) Global alignment, measuring the directional similarity between the descriptor and its global feature, i.e., the aver- age semantic embedding across all related 2D semantics; and (2) Internal compactness, reflecting the consistency of semantics within the descriptor's feature cluster. During inference, LaGa selects the highest weighted response of descriptors as the object's final output, thereby preserving
Semantic-Aware Scene Understanding. Prior to open- vocabulary methods, research primarily focus extend- ing 2D closed-set perception models to radiance fields. Semantic-NeRF (Zhi et al., 2021) explore semantic propa-
2

---

Tackling View-Dependent Semantics in 3D Language Gaussian Splatting
gation in NeRFs, while methods such ObSuRF (Stelzner
et al., 2021), DM-NeRF (Bing et al., 2023), Panoptic- NeRF (Fu et al., 2022), PCF-Lift (Zhu et al., 2024), NESF (Vora et al., 2022), and Siddiqui et al. (2023) employ 2D vision models for instance or panoptic segmentation in
19pl
C(p) = Cg
II a agg)
(I)
where cg.' is the color of g,: and Og. is computed from the corresponding 2D Gaussian with covariance Σ, scaled by a learnable per-Gaussian opacity.
3D. Additionally, Instance-NeRF (Liu et al., 2023b) and NeRF-RPN (Hu et al., 2023) introduce end-to-end models for instance detection within radiance fields. However, these methods are inherently constrained to closed-set categories.
To extend 3D-GS for scene understanding, many studies propose augmenting 3D Gaussians with additional attributes. A common approach is to attach a feature vector fg to each Gaussian g. This allows rendering a feature map using the same alpha blending formulation as in Equation (1):
Recent studies incorporate vision-language models for open- vocabulary scene understanding, LERF (Kerr et al., 2023) learns a feature field that mimics CLIP (Radford et al., 2021) features across multiple views, while 3D-OVS (Liu
et al., 2023a) performs weakly-supervised segmentation in NeRFs. With the emergence of 3D-GS, methods such as LangSplat (Qin et al., 2024), LEGaussians (Shi et al., 2024), GOI (Qu et al., 2024), and N2F2 (Bhalgat et al., 2024) focus on efficiently encoding high-dimensional language features within the explicit 3D Gaussians using feature com- pression techniques like codebooks or hyperplane-bait based grid decomposition. However, these approaches rely on 2D) rendered feature maps for perception, and show degraded performance in 3D scene understanding.
pppl Σ
F(P)
f G II ((g-
(2)
Specifically, language-embeddeed 3D Gaussian Splatring methods (Qin et al., 2024; Zhou et al., 2024; Bhalgat et al., 2024; Qu et al., 2024; Shi et al., 2024) align the rendered feature map F with CLIP's visual features during training. At inference, veee-vocabulary understanding is performed by computing the relevance between a text query and the rendered feature map. However, these learned features are optimized for 2D perception. When directly applied 3D understanding using the learned 3D features (fg g 9), performance degrades significantly, as shown in Figure 1.
To address this limitation, OpenGaussian (Wu et al., 2024b) and Open3DRF (Lee et al., 2025) constrain 3D features directly instead of supervising 2D rendered features. Open- Gaussian employs scene decomposition for 3D point-wise perception but primarily enforces semantic consistency within objects. However, overlooks view-dependent se- mantic variations and rigidly assigns semantics, leading to significant information loss and limiting its ability to achieve comprehensive 3D understanding. In contrast, we leverage scene decomposition to establish cross-view seman- tic connections, effectively capturing and preserving view- dependent semantics, thereby bridging the gap between direct 3D understanding and 2D pixel-wise understanding.
4. View-Dependency of 3D Semantics
As discussed in Section 1, a 3D object's semantics vary with viewpoint shifts. To quantitatively analyze this effect, we conduct two experiments. The data preprocessing for them is introduced in Section 5.2, which delivers set of 2D masks and their corresponding semantic features. Using these masks, the 3D-GS scene is decomposed into struc- turally meaningful but semantic-agnostic 3D objects Sec- tion 5.3, establishing connections between 2D masks and their corresponding 3D Gaussians. We assume the retrieved 3D Gaussians represent 'complete' 3D object.
3. Preliminary
3D Gaussian Splatting (3D-GS) is recent advancement in radiance fields that represents a 3D scene using a set of colored 3D Gaussians, g = {g1, {gi.g..... ......, ggN. }. where N the number of Gaussians in the scene. Each Gaussian's 's mean defines its position, and its covariance determines its scale. By "splatting" these 3D Gaussians onto image plane, 3D-GS enables real-time rendering.
We then introduce two analytical experiments: (1) Semantic similarity distribution analysis and (2) Semantic retrieval integrity analysis.
Semantic Similarity Distribution. After the 3D decom- position phase, the multi-view 2D masks along with the corresponding semantics of the same 3D object are clus- tered together. Thus, we can compute the cosine similarities of semantic features of the same object (intra-object) and between different objects (inter-object). This analysis is con- ducted across four scenes from the LERF (Kerr et al., 2023) dataset. The results are shown in Figure 3, where the two distributions demonstrate a significant overlap, indicating
Given a viewpoint with camera intrinsics and extrinsics, 3D-GS first projects the 3D Gaussians onto 2D plane. The color Cp) a pixel p is computed via alpha blending over an ordered set of Gaussians Gp overlapping the pixel. Let g, denote the i-th Gaussian in Gp. the pixel color is:
3

---

Tackling View-Dependent Semantics in 3D Language Gaussian Splatting
that single object's Gaussians may exhibit varying 2D semantics, highlighting the need for multi-view semantic aggregation in 3D understanding.
3.5%
Iatra-Objoct Cosine Inter-Ubject Cosine
3.0%
2.5%
5. Method
Percenagge
2.0%
In this section, we introduce the overall pipeline of LaGa (Section 5.1) and its key components, including 3D scene decomposition (Section 5.3) and view-aggregated semantic representation (Section 5.4).
1-5%
1.0%
0.5%
5.1. Overall Pipeline
0.0%
0.0
0.2
0.4
0.6
0.8
1.0
Costbe Similarity
Figure 5 presents the overall pipeline of LaGa. First, LaGa performs 3D scene decomposition to establish cross-view semantic connections by grouping multi-view 2D semantics corresponding to the same 3D object. This process natu- rally partitions the scene into 3D objects, which serve as fundamental carriers of multi-view semantics.
Figure 3. Distribution of intra- and inter-object cosine similarities.
that the multi-view semantics of single 3D object often ex- hibit lower cosine similarities than those of semantics from different 3D objects. This statistical finding provides strong evidence of the existence of dependett semantics.
Next, LaGa constructs view-aggregated semantic represen- tations to effectively preserve view-dependent semantics. It first adaptively extracts representative sets of semantic descriptors for 3D objects, consolidating multi-viww seman- tics into a more holistic representation. Then, a weighted descriptor relevance aggregation strategy refines the impor- tance of each descriptor, enhancing noise tolerance. Using these weighted descriptors, LaGa enables direct 3D scene understanding without relying on 2D rendered feature maps.
7.0%
6.0%
5.0%
Percentage
4.0%
Low / (Low High 50.2%
5.2. Data Preparation
3.0%
We follow the data preparation pipeline introduced by Qin et al. (2024) to obtain 2D segmentation and semantics priors. Concrete extraction method is introduced in Appendix B. After the data preparation, we have:
00D
Low Recall Semantics 46.6%
High Recall Semsnticc 46.3%
2.0%
Sonantics
7.1%
1.0%
0.0%
0.2
Recall
0.6
0.8
| N1) for each image I the training set I.
set of 2D masks M = {M} 10,1]
Figure 4. Distribution of recall rates for 2D semantic features.
Iz-normalized feature v E RCC corresponding to each mask M, where C denotes the feature dimension.
Semantic Retrieval Integrity. We project multi-view 2D semantics onto 3D Gaussians and use them to re- trieve 3D Gaussians from the scene (implementation details in Appendix B.3). Since each 2D mask corresponds to a "complete" 3D object via scene decomposition, the recall rate- prop-poootiion of semantically retrieved 3D Gaussians within the corresponding region-evaluate, how well the semantics capture the object. We define detected Gaussians as those with cosine similarity above 0.75 to given 2D feature. Results in Figure 4 show that low-recall features ([0.1, 0.9]) account for 50%. Notably, for complex scenes containing numerous objects with 360 viewpoints, such as 'Figurines', this percentage rises to 61.9% %. This indicates
5.3. 3D Scene Decomposition
To address the view-dependency of 3D semantics, we aim to establish connections among different 21) semantics de- picting the same 3D object. Since 2D semantic features are linked to multi-view 2D masks, and these masks inherently correspond coherent 3D objects, these connections can be effectively constructed by decomposing the 3D scene.
Inspired by existing 3D-GS decomposition methods (Kim et al., 2024; Ying et al., 2024; Lyu et al., 2024; Cen et al., 2025a), we adopt a contrastive learning approach to train a set of Gaussian affinity features F = (fg RC Ig e 9). where C' denotes the affinity feature dimension. These

---

Tackling View-Dependent Semantics in 3D Language Gaussian Splatting
(a) 3D Seene Decomposition
(b) View-Aggregated Semantic Representation
(b.1) Cross- View Descriptor Extraction
(b Weightdd Descriptor Relovance Aggregation
Multi-View
Sananti Desoritios
Global Senautics (Hq. (12))
Text Proupt
CLIP
Training Z
3D Gauasians with Affinity Features
Reader
SAM
Mf
go
St
D
d2
d
Adaptive K-Means (Alg. Al)
Descrigtor Weightts "pRR
Vanilla Relevance Reil Mel(d.g 4)
Masks.M
Contresive Training (Fis (3) (4)
RELAO (i)
Mask
S
DPP
Final Relevance
Clustaning
2 Weighted Relevance
(9)
g
Learned Affinity Features
Diffioemiible
Supervision
Scene Decorsponition (Eqs. (5) and (6))
Cosine Similarity
Maok (Semontic)M Me Groups
Decomposed 3D Secme
Masked Avenge Pooling
Figure 5. Overall pipeline of LaGa. LaGa first establishes cross-view semantic connections through contrastive 3D scene decomposition and then constructs new-aggregated semantic representations by adaptively clustering semantic descriptors and reweighting them.
features serve as indicators of whether two 3D Gaussians belong to the same object, ensuring that their similarity reflects their structural and spatial coherence.
Beyond connecting masks across views, mask prototypes also serve references for identifying the 3D Gaussians associated with their corresponding object. The prototype of each 3D object is computed as:
To train these affinity features, we render a feature map Fí for each view using Equation (2). We then assign a mask affinity prototype f CRC to each 2D mask M through masked average pooling:
fM
MES,
The object index to which a Gaussian g is assigned is determined by maximizing the similarity between its affinity feature fg and the prototype t. of each object S;:
MAP (M. F1) = Epes()) (p)
Σ (P) PEń(I)
(3)
arg max (gg. t.:).
(6)
Here, 8(I) denotes the set of pixels in image I. The train- ing objective encourages features within the same mask to cluster while separating those outside the mask:
where (.))) represents the cosine similarity.
At this stage, the scene is decomposed into Np distinct objects, represented as G U gs. Each object gs: corresponds group of 2D masks S, effectively preserv- ing multi-view semanties across the scene. Moreover, these objects enable the 3D Gaussians within each object to share common set of semantics, significantly reducing storage requirements and accelerating inference. In the next section, we detail the process of extracting semantic descriptors for these objects based on their related semantics.
L E (1-2(()). (fM, Fr(p)), IEI MEM PEб())
(4) Though the loss is computed in 2D, after training, the affinity features and mask prototypes of the same 3D object will converge into a compact cluster, as the rendered features originate from the same group of 3D Gaussians.
Let M = Usez MM be the set of masks of all training im- ages. After training, we employ HDBSCAN (McInnes et al.,
Why Is 3D Scene Decomposition Unaffected by View- Dependency? Unlike high-level semantics, which need to encode rich semantic information, 2D segmentation pri- marily captures object boundaries and thus remains stable across viewpoints. Except in extreme cases, SAM reliably produces accurate segmentations, allowing LaGa to estab- lish robust multi view semantic connections.
2017) to automatically cluster the masks in M based on their prototypes fM. resulting in M = U St, where S; is a set of 2D masks that belong to the same 3D object and N, denotes the number clusters. This step explicitly estab- lishes cross-vies view connections among 2D masks, grouping mult semanties into coherent 3D objects.
5

---

Tackling View-Dependent Semantics in 3D Language Gaussian Splatting
5.4. View-Aggregated Semantic Representation
Directional Consistency measures the cosine similar- ity between each descriptor and the global feature v of its corresponding object. Descriptors that aligns with the dominant semantics of the object are assigned higher weights. For instance, the spine of a book may appear as a "knife" from certain viewpoints, which is suppressed due to semantic inconsistency. In contrast, descriptor resembling a "passport" is more semanti- cally aligned with the global semantics "book," and is thus assigned higher weights.
For each object o with multi-view 2D masks S we de- note its multi-view semantic features as:
Dt: = (m M E S.).
LaGa obtains a robust 3D view-aggregated semantic repre- sentation from these multi-view 2D semantics through two key steps: 1) Cross-view descriptor extraction: Generates an informative set of representative semantic descriptors; 2) Weighted descriptor relevance aggregation: Adjusts the importance of each descriptor to mitigate noise.
Internal Compactness quantifies intra-cluster seman- tic agreement via the L2 norm of a descriptor. Semanti- cally consistent descriptors have compact clusters and yield higher norms. In contrast, if the features of de- scriptor are inconsistent with diverse directions, their vector average will cancel out, resulting in a lower norm. Thus, the norm serves confidence measure for semantic reliability.
Cross- View Descriptor Extraction. After the 3D scene decomposition, ys: maintain all available multi-view se- mantic information about the 3D object go; it is crucial to extract a representative set of descriptors that effectively summarize the semantic variations observed across different viewpoints. To achieve this, we apply K-means clustering to ys,, obtaining a set of cluster centroids:
Together, these criteria allow LaGa to suppress unreliable descriptors. Since the view-aggregated semantic represen- tations are assigned to 3D objects, LaGa enables direct 3D understanding without relying on rendered feature maps.
DEG = d EdeED RRR
=f
C" lie {1, ii
(8)
These centroids serve as semantic descriptors for the object gs. Since semantic complexity varies across objects, we determine the number of descriptors NG adaptively using the silhouette score (see Algorithm I for details).
6. Experiments
In this section, we first introduce the datasets and evaluation protocol. We then present both quantitative and qualitative results to demonstrate the effectiveness of LaGa. Addition- ally, we conduct extensive ablation studies to analyze the impact of different design choices in LaGa.
Weighted Descriptor Relevance Aggregation. Instead of treating all descriptors equally, we introduce a weighting mechanism that prioritizes more reliable descriptors. Given text query q E RC encoded by CLIP, we define the object- level relevance score as:
6.1. Datasets and Evaluation Protocol
REL .)) max wa Rel q).
(9)
d
We evaluate LaGa on LERF-OVS (Kerr et al., 2023; Qin et al., 2024), 3D-OVS (Liu et al., 2023a), and ScanNet (Dai et al., 2017). LERF-OVS consists of complex 360 indoor scenes, while 3D-OVS features forward-facing scenes with long-tailed categories. Both datasets provide 2D annota- tions. Unlike prior methods that perform segmentation on rendered 2D feature maps, we conduct 3D segmentation at the Gaussian level, generating binary Gaussian segmenta- tion maps ('1' for foreground, "0" for background). These maps then rendered into 2D views and evaluated against ground-truth masks.
The relevance score Rd,(d, q) follows Kerr et al. (2023):
exp (d.q)
Rel (d, q) = min exp (d, q) + exp (d )
(10)
where di represents canonical text embeddings'.
The weight a for each descriptor d is based on two criteria:
S.S.
w d.
| 1|2
For ScanNet, we follow OpenGaussian (Wu et al., 2024b) (11) to perform 3D point cloud semantic segmentation, directly comparing predictions with point-wise ground-truth annota- tions for a more direct assessment of 3D perception quality.
d
|dl|
||||
S. 11
(ii) Internal Compactness
Directional Consistency
S. = 1AS.
Σ v,
(12) 6.2. Quantitative Results
V
Following prior work, the canonical phrases are "object," "thing," " "texture," " and "stuff".
LERF-OVS Dataset. Table I compares LaGa with both 3D and 2D approaches, demonstrating substantial improve-
6

---

Tackling View-Dependent Semantics in 3D Language Gaussian Splatting
Table Quantitative mloU (%) results on the LERF-OVS dataset: "F." (Figurines), "T." (Teatime), "R." (Ramen), and "W." (Waldo Kitchen). * indicates results from Wu (2024b), * indicates reimplementation, and denotes unrefereed preprints.
Table 2. Quantitative mloU (%) results the 3D-OVS dataset.
Methods marked with denote concurrent preprints.
METHODS
BED
BENCH
ROOM
SOFA
LAWN
MEAN
LERF
73.5
53.2
46.6
27.0
73.7
54.8
METHODS
F.
T.
R.
W.
MEAN
3D-VVS
89.5
89.3
92.8
74.0
88.2
86.8
LSEG
7.6
21.7
7.0
29.9
16.6
GOI
89.4
92.8
91.3
85.6
94.1
90.6
LERF
38.6
45.0
28.2
37.9
37.4
LEGAUSSIANS
84.9
91.1
86.0
87.8
92.5
88.5
LEGAUSSIANS
60.3
44.5
52.6
41.4
46.9
LANGSPLAT
92.5
94.2
94.1
90.0
96.1
93.4
3D
LANGSPLAT
44.7
$1.2
44.5
51.4
N2F2
93.8
92.6
93.5
92.1
96.3
93.9
N2F2
47.0
56.6
47.9
54.4
OCCAMI.GS*
96.8
95.8
96.5
88.8
97.0
95.0
OCCAMLGSS
58.6
51.0
65.3
61.3
VLGS*
96.8
97.3
97.7
95.5
97.9
97.1
VIGS*
58.1
73.5
61.4
54.8
62.0
LAGA (OURS)
96.8
92.8
97.0
93.0
96.9
95.3
OPENGAUSSIAN
39.3
60.4
31.0
22.7
38.4
SAGA
36.2
19.3
53.1
14.4
30.7
3D
LANGSPLAT
25.9
35.6
29.3
33.5
31.1
LEGAUSSIANS
31.2
34.5
17.6
17.3
25.2
Table Quantitative results on the ScanNet dataset.
OPENGAUSSIAN
61.1
59.1
29.2
31.9
45.3
SUPERGSEG*
43.7
55.3
18.1
26.7
35.9
19 CLASSES 15 CLASSES MIOU/ MÁCC.
10 CLASSES MIOU MACC.
LAGA (OURS)
64.1
70.9
55.6
65.6
64.0
METHODS
MIOU / MACC.
LEGAUSSIANS
10.9
22.2 5.4/
12.8 28.6
LANGSPLAT
3.8/9.1
8.4
OPENGAUSSIAN LAGA (OURS)
24.7/41.5
30.1/48.3
/
ments over previous tofte--o--hh-arr methods. Unlike 2D methods that rely on rendered feature maps, LaGa performs direct 3D segmentation at the Gaussian level. Under identi- cal settings, LaGa achieves an 18.7% mloU improvement OpenGaussian (Wu et al., 2024b)
32.5/49.1
35.5/ 53.5
42.6 / 63.2
The results are shown in Table 3. Predictably, both 2D segmentation methods, LangSplat (Qin et al., 2024) and LEGaussians (Shi et al., 2024), fail to produce reasonable results due to poor 3D point feature quality. In contrast, LaGa outperforms OpenGaussian, demonstrating superior capability in handling point cloud segmentation.
Notably, LaGa achieves significant gains in the "Waldo Kitchen" scene, where OpenGaussian's rigid feature as- signment fails to capture key distinguishing semantics that appear inconsistently across viewpoints. This result high- lights the importance of LaGa's riew-aggregated semantic representation in addressing view-dependent semantics.
Difference between LaGa and OpenGaussian. Both OpenGaussian (Wu et al., 2024b) and LaGa decompose 3D scenes for open-vocabulary understanding. However, Open- Gaussian assigns a 2D CLIP feature to each Gaussian via a rule-based representative view selection, which overlooks the information in multi-view semantics and results in sub- optimal performance. In contrast, LaGa extracts semantic descriptors through adaptive multi-view clustering and ap- plies weighted relevance aggregation to suppress noise and enhance robustness. This design enables LaGa to achieve a significantly more robust 3D semantic representation, with an improvement of + +18.7% 7% in mloU.
Furthermore, LaGa outperforms recent preprints, including OccamLGS (Cheng al., 2024), VLGS (Peng al., 2025), and SuperGSeg (Liang et al., 2024), establishing a new benchmark for open-vocabulary 3D segmentation.
3D-OVS Dataset. As shown in Table 2, LaGa achieves competitive 3D segmentation performance on the 3D-OVS dataset, reaching 95.3% mloU, comparable state-of- the-art 2D methods. The lack of significant improvement can be attributed to the dataset's lower complexity-ees each scene contains only few distinctive objects, and existing methods already achieve near-optimal performance (over 90% mloU). Additionally, the forward-facii facinf scene config- uration and limited viewpoint variations reduce the impact of view-dependent semantics, which LaGa is designed to handle. These results align with expectations.
6.3. Qualitative Results
We provide visual comparison with existing methods in Figure Compared 2D methods such as LEGaus- sians (Shi al., 2024) and LangSplat (Qin et al., 2024). LaGa generates sharper boundaries and more complete 3D segmentations. While these prior methods can coarsely de- tect objects, they suffer from multiple false positives and false negatives due to the inherent limitation of learning 3D Gaussian features from restricted viewpoints.
ScanNet Dataset. The detailed implementation details of point cloud segmentation can be found in Appendix B.6.
We revise the evaluation script of OpenGaussian's official code. As result, our reimplementation achieves +6.9% mloU higher than originally reported.
For example, in the "Waldo Kitchen- Ottolegh" example,

---

Tackling View-Dependent Semantics in 3D Language Gaussian Splatting
Prompts:
Prompis:
Prompts:
Prompts:
"Mifly" "Spatula"
"Eige"
"Three cookies"
"Toaster"
"Kamaboko"
"Ten glass"
"Ortolenghi"
Figurines
Ramen
Teatime
Waldo kitchen
LaGa (ours)
LEGeussions
LangSplat
OpenGGussian
Figure 6. Qualitative comparison on the LERF-OVS dataset. LaGa delivers more precise 3D segmentations within 3D-GS. Although corresponding views are provided with colored bounding boxes clarity, neither viewpoint information nor visual prompts are used.
both LEGaussians and LangSplat capture only partial outline of the book, as CLIP can only recognize its title from certain perspectives. In most views, the title unidentifiable, leading incomplete feature fusion and missing regions. contrast, LaGa effectively captures the book's 's full semantics by integrating information across multiple views.
MIP- 160-Oneen
Text Prompts:
"Football"
"Vase"
"Dable"
"Gimss"
"Door"
OpenGaussian (Wu et al., 2024b) also produces clean and distinct segmentations by enforcing feature consistency within pre-segmented 3D regions. However, it fails to ac- count for view-dependent semantics, as it rigidly assigns single 2D feature an entire 3D region. This limitation results in frequent misclassifications (e.g., Figurines- Miffy, Ramen-Egg, and Waldo Kitchen- Toaster).. These findings further underscore the necessity of preserving multi-view semantics and highlight the effectiveness of LaGa.
MIP 360 Rovors
Text Prompts:
"Speake""
"Slippers"
"Carpet"
"Bottle"
"Basin"
Figure 7. Visualization results on MIP-360 dataset. LaGa shows accurate localization across diverse textual prompts both indoor and outdoor complex environments.
To further demonstrate the generalizability of LaGa, we present visualization results on the MIP-360 dataset (Barron al., 2022). illustrated in Figure 7, LaGa performs well in both complex indoor and outdoor environments. For more qualitative results, including visual comparisons, multi-view and multi-granularity segmentation results, vi- sualizations on the 3D-OVS dataset, and examples of mask clustering and scene editing, see Appendix C.
gregation. To assess the impact of cress-view descriptor extraction, we introduce two simple baseline methods for deriving semantic descriptors of 3D objects:
1. Average Pooling: Represent the object GS. by com- puting the average of the visual features in yS:
2. Max Pooling: Retain all features in ys, and, during inference, use the highest response among them as the representative response for the object go.
6.4. Ablation Study
Our ablation studies evaluate the effectiveness of the view-
aggregated semantic representation, focusing on cross-view descriptor extraction and weighted descriptor relevance ag-
The results are presented in Table 4. "Fixed: k" denotes clustering the multi-view features using a fixed number of
8

---

Tackling View-Dependent Semantics in 3D Language Gaussian Splatting
Table Ablation study on the LERF-OVS dataset. For cach scene. the best and second-best results are highlighted in bold and underlined fonts. "DW" represents the descriptor weighting scheme introduced in Section 5.4. Within each "DW" experiment group, superior results are marked with gray background.
Bag-of- -orrds Effect in CLIP:
METHODS
F.
T.
R.
W.
MEAN
AVG. POOLING
48.6
64.9
47.8
52.4
53.4
"Pirate hat on the rubber duck"
"Cookies in the plate"
MAX POOLING
38.4
42.7
35.4
56.7
43.3
Lack of Context 2D Semantics:
-DW
64.7
58.9
50.5
62.2
59.1
FIXED: 5
+ DW
63322
64.5
56.2
63.5
61.9
DW
62.8
68.0
46.3
60.3
59.4
FIXED: 10
+ DW
59.5
71.3
54.3
63.9
62.3
DW
61.1
63.5
41.6
63.0
57.3
FIXED: 20
+ DW
58.2
63.2
48.9
66.4
59.2
"Chair under the shoep"
"Hooves"
DW
59.7
65.5
53.8
62.8
60.4
+ DW
61.6
69.3
53.6
59.2
60.9
ADAPTIVE
Figure 8, Failure cases of LaGa, with prompts shown below and predicted regions highlighted.
+ DW 60.8
67.3
55.6
63.9
61.9
+ DW
64.1
70.9
55.6
65.6
64.0
Bag-of- Words Effect in CLIP. Prompts such as "pi- rate hat on the rubber duck" or "cookies in the plate" require models to resolve compositional semantics. However, CLIP tends to activate isolated nouns (e.g., "hat", "duck") rather than the full phrase. LaGa inherits this limitation from CLIP.
clusters k, and "Adaptive" represents our final descriptor extraction method, which adaptively selects the number of clusters. "DW" denotes the descriptor weighting scheme.
By replacing the descriptor extraction method with the naive "average pooling" and "max pooling" strategies, the results show significant drops of 10.6% mIoU and 20.7% mIoU, respectively. This is because that merely averaging the multi-view features can lead to information loss due view- dependent semantics. Conversely, retaining all features makes the segmentation susceptible to noisy features ex- tracted by CLIP. In contrast, using clustering algorithm to preserve multi-view semantic information leads to a notice- able performance improvement. However, is evident that the optimal number of clusters varies across different scenes due differences in data distribution. Manually adjust- ing these hyper-parameter is impractical and unreasonable, demonstrating the necessity of our adaptive strategy.
Lack of Context in 2D Semantics. LaGa extracts 2D semantics by feeding SAM-segmented image crops into CLIP to reduce distractions from unrelated content. However, this operation removes necessary contextual information, leading to errors in relational ("chair un- der the sheep")) or part-level queries ("hooves").
These failure cases highlight the gap between real-world opeabuoccbylary perception and current model capabilities. Future work could explore the use of large language models to better handle such compositional grounding
Moreover, regardless of whether a fixed number of clus- ters or the adaptive strategy is used, results with the de- scriptor weighting consistently outperform those without it. DW and DW represent descriptor weighting schemes that consider only the internal compactness of the descriptor ([d||2) and the global alignment (/d, v-S)),, respectively. Both schemes show improvements compared to the -DW setting: however, combining them yields the most significant enhancement, highlighting the necessity of our design.
7. Conclusion
In this paper, we investigate language-driven open- vocabulary scene understanding methods based on 3D-GS and identify the view-dependency of 3D semantics as a crit- ical limitation hindering robust 3D understanding. Through experiments, we validate the existence and impact of this is- sue. To address this challenge, we propose LaGa, method that establishes cross-view connections among multi-view semantics and construets view-aggregated semantic repre- sentations for 3D scene understanding. Extensive experi- ments demonstrate that LaGa effectively achieves a more comprehensive understanding of 3D scenes. Our findings provide a new perspective for advancing vocabuaayyl scene understanding within the 3D-GS framework.
For more discussion on the hyper- parameter selection of LaGa, please refer to Appendix B.5.
6.5. Discussion on Failure Cases
We analyze common failure cases of LaGa. As shown in Fig-
ure two main issues are observed:

---

Tackling View-Dependent Semantics in 3D Language Gaussian Splatting
Acknowledgments
Chen, Y., Chen, Z., Zhang, C., Wang, F., Yang, X., Wang, Y., Cai, Z., Yang, I.,, Liu, H., and Lin, G. Gaussianeditor: Swift and controllable 3d editing with gaussian splatting. In CVPR, 2024.
This work was supported by NSFC 62322604, NSFC 62176159, Shanghai Municipal Science and Technology Major Project 2021SHZDZX0102.
Cheng, J., Zaech, J.-N., Gool, L. V., and Paudel, D. P. Oc- cam 's Igs: A simple approach for language gaussian splat- ting. arXiv preprint arXiv:2412.n 2xxi:2412.01807, 2024.
Impact Statement
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.
Dai, A., Chang, A. X., Savva, M., Halber, M., Funkhouser, T., and Nießner, M. Scannet: Richly-annotated 3d recon- structions of indoor scenes. In CVPR, 2017.
Duan, Y., Wei, F., Dai, Q.. He. Y., Chen, W., and Chen, B. 4d-rotor gaussian splatting: Towards efficient novel view synthesis for dynamic scenes. In SIGGRAPH, 2024.
References
Barron, J. T., Mildenhall, B., Tancik, M., Hedman, P., Martin-Brualla R., and Srinivasan, P. P. Mip-nerf: A multiscale representation for anti- anti-aliasing neural radiance fields. In ICCV, 2021.
Fan, Z., Wang, P., Jiang, Y., Gong, X., Xu, D., and Wang, Z. Nerf-sos: Any-view self-supervised object segmentation on complex scenes. ICI.R, 2023.
Barron, J. T., Mildenhall, B., Verbin, D., Srinivasan, P., and Hedman, P. Mip-nerf 360: Unbounded anti-aliased neural radiance fields. In CVPR, 2022.
Kriiovich-Keil, S., Yu, A., Tancik, M., Chen, Q.. Recht, B., and Kanazawa, A. Plenoxels: Radiance fields without neural networks. In CVPR, 2022.
Barron, J. T., Mildenhall, B., Verbin, D., Srinivasan, P. P., and Hedman, Zip-nerf: Anti-aliased grid-based neural radiance fields. ICCV, 2023.
Fu, X., Zhang, S., Chen, T., Lu, Y., Zhu, L., Zhou, X., Geiger, A., and Liao, Y. Panoptic nerf: 3d-to-2d label transfer for panoptic urban scene segmentation. In 3DV, 2022.
Bhalgat, Y., Laina, I., Henriques, J. a. F., Vedaldi, A., and Zisserman, A. Contrastive lift: 3d object instance seg- mentation by slow-fast contrastive fusion. In NeurIPS, 2023.
Goel, R., Sirikonda, D., Saini, S., and Narayanan, Inter- active segmentation of radiance fields. In CVPR, 2023.
Hedman, Srinivasan, P. P., Mildenhall, B., Reiser, C., Barron, J. T., and Debevec, P. Baking neural radiance fields for real-time view synthesis. IEEE TPAMI, 2024.
Bhalgat, Y., Laina, I., Henriques, J. F., Zisserman, A., and Vedaldi, A. N2f2: Hierarchical scene understanding with nested neural feature fields. In ECCV, 2024.
Hu, B., Huang, J., Liu, Y., Tai, Y.W.,, and Tang, C.-K. Nerf- rpn: A general framework for object detection in nerfs. In CVPR, 2023.
Bing, W., Chen, L., and Yang, B. Dm-nerf: 3d scene geom- etry decomposition and manipulation from 2d images. In ICLR, 2023.
Hu, X., Wang, Y., Fan, L., Fan, J., Peng. J., Lei, Z., Li, Q. and Zhang, Z. Segment anything in 3d gaussians. arXiv preprint arXiv:2401.17857, 2024.
Caron, M., Touvron, H., Misra, I., Jégou, H., Mairal, J., Bojanowski, and Joulin, A. Emerging properties in self-supervised vision transformers. In ICCV, 2021.
Kerbl, B., Kopanas, G., Leinkühler, T., and Drettakis, G. 3d gaussian splatting for real-time radiance field rendering. ACM TOG, 2023.
Cen, J., Zhou, Z., Fang, J., Yang, C., Shen, W., Xie, L., Jiang, D., Zhang, X., and Tian, Q. Segment anything in 3d with nerfs. In NeurlPS, 2023.
Kerr, J., Kim, C. M., Goldberg, K., Kanazawa, A., and Tancik, M. Lerf: Language embedded radiance fields. In ICCV, 2023.
Cen, J., Fang, J., Yang, C., Xie, I.., Zhang, X., Shen, W., and Tian, Q Segment any 3d gaussians. In AAAI, 2025a.
Kim, C.M Wu, M., Kerr, J., Tancik, M., Goldberg, K., and Kanazawa, A. Garfield: Group anything with radiance fields. CVPR, 2024.
Cen, J.. Zhou, Z., Fang, J., Yang, C., Shen, W., Xie, L., Jiang, D., Zhang, X., and Tian, Q. Segment anything in 3d with radiance fields. IJCV, 2025b.
Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C.,
Chen, A., Xu, Z., Geiger, A., Yu, J., and Su, H. Tensorf: Tensorial radiance fields. In ECCV, 2022.
Gustafson, L., Xiao, T., Whitehead, S., Berg, A. C., Lo, W.- Y., et Segment anything. In ICCV, 2023.
10

---

Tackling View-Dependent Semantics in 3D Language Gaussian Splatting
Kobayashi, S., Matsumoto, E., and Sitzmann, V. Decom- posing nerf for editing via feature field distillation. In NeurIPS, 2022.
radiance fields using depth oracle networks. In Computer Graphics Forum, 2021.
Peng, Q. Planche, B., Gao, Z., Zheng, M., Choudhuri, A., Chen, T., Chen, C., and Wu, Z. 3d vision-language gaussian splatting. ICLR, 2025.
Langley, P. Crafting papers on machine learning. Langley, P. (ed.), Proceedings of the 17th International Conference on Machine Learning (ICML 2000), pp. 1207-1216, Stan- ford, 2000. Morgan Kaufmann.
Qin, M., Li, W., Zhou, J., Wang, H., and Plister, H. Langsplat: 3d language gaussian splatting. In CVPR, 2024.
Lee, H., Yun, Y., Bae, J., Kim, S., and Uh Y. Rethinking open-vocabulary segmentation of radiance fields in 3d space. In AAAI, 2025.
Qu, Y., Dai, S., Li, Lin, J., Cao, L., Zhang, S., and Ji, Goi: Find 3d gaussians of interest with an optimizable open open-oocabulary semantic-space hyperplane. In ACM MM, 2024.
Liang, S., Wang, S., Li, K., Niemeyer, M., Gasperini, S., Navab, N., and Tombari, F. Supergseg: Open-vocabulary 3d segmentation with structured super-gaussians. arXiv preprint ar arXiv:24/2. 10231, 2024.
Radford, A., Kim, J. W. Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., and Sutskever, I. Learning transferable visual models from natural language supervision. In ICML, 2021.
Lindell, D. B., Martel, N. P., and Wetzstein, G. Autoint: Automatic integration for fast neural volume rendering. In CVPR, 2021.
Liu, K., Zhan, F., Zhang, J., XU, M., Yu, Y., Saddik, A. E., Theobalt, C., Xing, E., and Lu, S. Weakly supervised 3d open- vocabulcyyy segmentation. In NeurlPS, 2023a.
Reiser, C., Peng, S., Liao, Y., and Geiger, A. Kilonerf: Speeding up neural radiance fields with thousands of tiny mips. In ICCV, 2021.
Liu, Y., Hu, B., Huang, J., Tai, Y.-W,, and Tang, C.-K. Instance neural radiance field. In 20C3.
Ren, Z., Agarwala, A., Russell, C.,, Schwing, G., and Wang. O. Neural volumetric object selection. In CVPR, 2022.
Lyu, W., Li, X., Kundu, A., Tsai, Y.-H,, and Yang, M.-H. Gaga: Group any gaussians via 3d-aware memory bank. arXiv preprint arXiv:2404.00 079977777777. 2024.
Shi, J.-C., Wang, M., Duan, H.-B., and Guan, S.-H. Lan- guage embedded 3d gaussians for open-vocabulary scene understanding. In CVPR, 2024.
Martin-Brualla, R., Radwan, N., Sajjadi, M. S., Barron, T., Dosovitskiy, A., and Duckworth, D. Nerf in the wild: Neural radiance fields for unconstrained photo collections. In CVPR, 2021.
Siddiqui, Y., Porzi, L., Buló, S. R., Müller, N., Nießner, M., Dai, A., and Kontschieder, P. Panoptic lifting for 3d scene understanding with neural fields. In CVPR, 2023.
McInnes, L.,, Healy, J., and Astels, S. hdbscan: Hierarchi- cal density based clustering. Journal of Open Source Software, 2017.
Stelzner, K., Kersting. K., and Kosiorek, A. R. Decom- posing 3d scenes into objects via unsupervised volume segmentation. arXiv preprint arXiv:2104.01148, 2021.
Mihajlovic, M., Prokudin, S., Tang, S., Maier, R., Bogo, F,, Tung. T., and Boyer, E. Splatfields: Neural gaussian splats for sparse 3d and 4d reconstruction. In ECCV, 2025.
Sun, C., Sun, M., and Chen, H. Direct voxel grid optimiza- tion: Super-fast convergence for radiance fields recon- struction. In CVPR, 2022.
Tang, J., Chen, Z., Chen, X., Wang, T., Zeng, G., and Liu, Z. Lgm: Large multi-view gaussian model for high- resolution content creation. In European Conference on Computer Vision, pp. 1-18. Springer, 2025.
Mildenhall, B., Srinivasan, P., Tancik, M., Barron, J. T., Ramamoorthi, R., and Ng, R. Nerf: Representing scenes as neural radiance fields for view synthesis. In ECCV, 2020.
Tschernezki, V., Laina, I., Larlus, D., and Vedaldi, A. Neural feature fusion fields: 3d distillation of self superviedd 2d image representations. 3DV, 2022.
Müller, T., Evans, A., Schied, C., and Keller, A. Instant neural graphics primitives with a multiresolution hash encoding. ACM TOG, 2022.
Vora, S., Radwan, N., Greff, K., Meyer, H., Genova, K., Saj- jadi, M. S., Pot, E., Tagliasacchi, A., and Duckworth, D. Nesf: Neural semantic fields for generalizable semantic segmentation of 3d scenes. R, 2022.
Neff, T., Stadlbauer, P., Parger, M., Kurz, A., Mueller, J. H., Chaitanya, C. R. A., Kaplanyan, A., and Steinberger, M. Donerf: Towards real-time rendering of compact neural

---

Tackling View-Dependent Semantics in 3D Language Gaussian Splatting
Wang, J., Fang, J., Zhang, X., Xie, L., and Tian, Q. Gaus- sianeditor: Editing 3d gaussians delicately with text in- structions. In CVPR, 2024.
Wizadwongsa, S., Phongthawee, P., Yenphraphai, J., and Suwajanakorn, Nex: Real-time view synthesis with neural basis expansion. In CVPR, 2021.
Wu, G., Yi, T., Fang, J., Xie, L., Zhang. X., Wei, W., Liu, W., Tian, Q. and Wang, X. 4d gaussian splatting for real-time dynamic scene rendering. CVPR, 2024a.
Wu, Y., Meng, J., Li, H., Wu, C., Shi, Y., Cheng, X., Zhao, C., Feng, H., Ding, E., Wang, J., and Zhang, Open- gaussian: Towards point-level 3d gaussian-based open vocabulary understanding. In NeurIPS, 2024b.
Yang. C., Li, S., Fang, J., Liang, R., Xie, L., Zhang, X., Shen, W., and Tian, Q. Gaussianobject: High-quality 3d object reconstruction from four views with gaussian splatting. ACM Transactions on Graphics (TOG), 2024a.
Yang, Z., Gao, X., Zhou, W., Jiao, S., Zhang, Y., and Jin, X. Deformable 3d gaussians for high-fidelity monocular dynamic scene reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 20331 1-20441, 2024b.
Yi, T., Fang, J., Wang, J., Wu, G., Xie, L.., Zhang, X., Liu, W., Tian, Q., and Wang. X. Gaussiandreamer: Fast generation from text to 3d gaussians by bridging 2d and 3d diffusion models. CVPR, 2024.
Ying, H., Yin, Y., Zhang, J., Wang, F., Yu, T., Huang, R., and Fang, L. Omniseg3d: Omniversal 3d segmentation via hierarchical contrastive learning. In CVPR, 2024.
Zhang, K., Riegler, G., Snavely, N., and Koltun, V. Nerf++: Analyzing and improving neural radiance fields. arXiv preprint arXiv:2010.07492, 2020.
Zhi, S., Laidlow, T., Leutenegger, S., and Davison, A. J. In-place scene labelling and understanding with implicit scene representation. In ICCV, 2021.
Zhou, S., Chang, H., Jiang, S., Fan, 7., Zhu, 7., Xu, D., Chari, P., You, S., Wang, Z., and Kadambi, A. Feature 3dgs: Supercharging 3d gaussian splatting to enable dis- tilled feature fields. In CVPR, 2024.
Zhu, R., Qiu, S., Wu, Q. Hui, K., Heng, P., and Fu, C. Pef- lift: Panoptic lifting by probabilistic contrastive fusion. In ECCV, 2024.
12

---

Tackling View-Dependent Semantics in 3D Language Gaussian Splatting
A. Appendix Overview
This appendix provides additional details and analyses to complement the main paper. It includes implementation details, experimental setup, and extensive qualitative results. The appendix structured follows:
Appendix B: Implementation details, including data preparation, additional training strategies, semantic retrieval analysis, cross-view descriptor extraction, nlper-paramette discussion and ScanNet experiments.
Appendix C.I: Additional qualitative comparisons the LERF-OVS dataset.
Appendix C.2: Multi-view segmentation results on the LERF-OVS dataset.
Appendix C.3: Multi-view segmentation results on the 3D-OVS dataset.
Appendix C.4: Multi-granularity segmentation results on the LERF-OVS dataset.
Appendix C.5: Scene editing examples.
Appendix C.6: Examples of mask groups obtained from 3D scene decomposition.
Appendix C.7: Analysis of view-dependent 3D semantics, demonstrating variations in 2D relevance scores across different viewpoints.
B. Implementation Details
To extract 2D segmentation priors and language features for the training images, we follow previous works (Qin et al., 2024; Bhalgat et al., 2024; Wu et al., 2024b) by using the ViT-H model of SAM and the OpenCLIP ViT-B/16 model of CLIP. The semantic feature dimension 512, and the Gaussian affinity feature dimension C" is 32, For the LERF-OVS dataset, follow LangSplat to train three-level model corresponding to the "subpart," "part," and "whole" levels of SAM-extracted masks. Rather than selecting the level with the highest response as in LangSplat, we average responses across all three levels, which we find to be more robust. For 3D-OVS and ScanNet, we use only the "whole" level of SAM masks, as these datasets are simpler. For each scene, the 3D-GS model is trained for 30000 iterations, followed by 30000 iterations training of the Gaussian affinity features. For ScanNet, we apply a -asedbased local feature smoothing operation following SAGA (Cen et al., 2025a) during training the affinity features. During inference, in addition to the relevance score, we find that applying an auxiliary cosine similarity threshold (0.23) helps remove unwanted regions. For all remained objects in the scene, relevance scores are first min-max normalized. A 3D bilateral filtering step then applied to the resulting 3D relevance map to suppress noise. Gaussians with relevance scores above 0.6 are classified as foreground". All experiments are conducted on single NVIDIA RTX 3090 GPU.
B.1. Concrete Data Preparation Pipeline
We follow LangSplat (Qin et al., 2024) to combine SAM with CLIP to generate 2D segmentation priors with pixel-wise semantic features. Specifically, for each 2D image I, we first use SAM to extract a set of 2D masks automatically, denoted by M = {M, E [0,1) i I,.... N1). To obtain the corresponding visual semantic feature for a mask M., we begin by computing its bounding box B E R4. We then apply M. to and crop the resulting masked image with B, producing a cropped image M Finally, we resize this crop 224 x 224 and feed it into the CLIP image encoder to obtain its visual feature MM!
B.2, Additional Training Strategy
Due to severe data imbalance when training Gaussian affinity features, we resample positive and negative samples in each training iteration. Rather than applying the loss function in Equation (4) directly, we split it into two components: a positive component that draws features within the same mask closer to the feature prototype fMr and negative component that pushes features from outside the mask away from the prototype. We randomly select a set of positive samples PM and negative samples M ensuring [pM] = [MM| The rebalanced loss is then defined as:
Thiis strategy follows LangSplat, which applies min-max normalization to the 2D relevance map and threshold for foreground- background separation. contrast, LaGa operates in and applies the same procedure to the 3D relevance map.
13

---

Tackling View-Dependent Semantics in 3D Language Gaussian Splatting
f, E E E (f)) IEI MEM! PEPM
Σ E Σ (1-M max Fr(p)).
(13)
IEZ MEM! PEAt
This re-balancing strategy is essential for obtaining high-quality scene decomposition results.
In addition, better align affinity features between inner Gaussians and those located near the object surface, follow SAGA (Cen et al., 2025a) and incorporate feature norm regularization. Specifically, the aggregated feature F(p) location p is computed as:
1ppl _
P R
F(p)
a B:
(1 - a g.
(14)
112
j=1
and the regularization term is defined as:
Coorm(P) =1 1
(15)
resulting in the final loss function:
C C Σ Σ Lnorm(p)
(16)
IET PEA(T)
Please refer to Cen et al. (2025a) for more details on the underlying mechanism of this regularization.
B.3. Details of Semantic Retrieval
To obtain the fused semantic features of 3D Gaussians, we first analyze the 3D Gaussians responsible for rendering each 2D mask. This is achieved by examining the weights used in the rendering phase (Equation (1)). For ease of implementation, given 2D mask M. we the backward pass of the differentiable rasterization algorithm to propagate the mask information onto the 3D Gaussians. This process yields of gradient scores Z {z g E G). Let lg denote the fused visual feature from the semantic retrieval experiment. During iteration of projection, 1g is updated follows:
lg lg
(17)
where M represents the feature of the 2D mask M. After processing all mask features, the fused visual features are normalized:
Lg
(18)
|1g l
Using these fused features, the 3D Gaussians corresponding to given 2D mask feature M are retrieved as:
{g g E (vM, 1g) 0.75}.
(19)
Note that the 0.75 threshold is chosen empirically based analysis of the precision-recall trade-off under different cosine similarity values. Here, precision refers to the proportion of retrieved 3D Gaussians that actually belong to the corresponding 3D object. Low precision indicates that unrelated Gaussians are being retrieved for given 2D mask.
As shown in Table 5, lowering the similarity threshold (e.g., to 0.7) increases the proportion of samples with high recall (20.9), but at the cost of significantly reduced precision. For example, at 0.7, the average precision of high-recall samples drops to just 13.4%.
From these observations, find that thresholds in the [0.75, 0.8) range strike better balance. We conservatively select 0.75, where 50.2% of samples exhibit low recall, clearly demonstrating the challenge of view-dependent semantics. A threshold of 0.8 also yields valid results, with fewer high-recall samples but higher precision.
Note that the average precision does not reach 100%, as CLIP features operate at the semantic level, not the instance level. Therefore, 3D Gaussians with similar semantics but belonging to different objects may also be retrieved, even with a high threshold.
14

---

Tackling View-Dependent Semantics in 3D Language Gaussian Splatting
Table 5. Precision-recall trade-off different relevance thresholds.
Threshold
0.5
0.6
0.7
0.75
0.8
0.9
Low / (Low+High) (%)
2.6
13.1
33.7
50.2
65.4
93.8
AP of High (%)
1.7
3.9
13.4
24.1
38.6
48.7
Algorithm Cross- View Descriptor Extraction
Input: Feature set ys: maximum clusters Kmax
Output: Semantic descriptors D for the 3D object Go, and the number of deseriptors No
Initialize best silhouette score s" -1 Initialize optimal number of clusters K 1 for K to Kmux do Perform K-means clustering on VS: with K clusters, obtaining clusters ,I....,Cr Compute silhouette score SK for the current clustering if sK > 8* then
+- sK: K K c+ 1.C
end if
end for
No9 f- K DgS + [CCentroid()),... [ Centroid(Cxk))
B.4. Detailed Algorithm of the Cross- Vross-View Descriptor Extraction
The pseudo code of the cross-view descriptor extraction is shown Algorithm 1.
B.5. Klyper-parameter Selection and Discussion
LaGa involves two key byper-parameters: the maximum number of clusters (Knax) in the adaptive K-means algorithm, and the cluster selection threshold used in HDBSCAN. Kmax is set to 20 for all experiments. For multi-level modeling (see Appendix B. 1),, e set to 0.1, 0.2, and 0.3 for the "subpart", "part", and "whole" levels, respectively. experiments with single-level decomposition, e fixed at 0.1.
To evaluate the robustness of LaGa with respect to these hyper-parameters, conduct ablation studies the LERF-OVS dataset. As shown in Table 6, LaGa maintains stable performance across wide range of Knax values (5-30). Similarly, Table 7 demonstrates that LaGa is insensitive to within the range of 0 to 0.3 at the "whole" level. However, setting e too large (e.g., 0.4) can lead to unintended object merging.
B.6. Implementation Details for Experiments on ScanNet Dataset
For the point cloud semantic segmentation task on the ScanNet dataset, we adapt our 3D segmentation method to align with the evaluation protocol, which does not support a complete 3D-GS model. Specifically, during the cross-view semantics grouping phase, we fix the coordinates of the 3D points (i.e., the means of the 3D Gaussians) and disable the densification mechanism of 3D-GS. This allows us to directly train the affinity features of these Gaussians while maintaining consistency in other processes. During evaluation, we employ the inference procedure described Section 5.4 to assign each 3D point multi-class segmentation score, classifying each point into the category with the highest score.
C. More Qualitative Results
In this section, we present additional visual results to further demonstrate the effectiveness of LaGa. We first show more qualitative comparisons with existing methods on the LERF-OVS dataset. Next, we provide multi-view 3D segmentation results on LERF-OVS to demonstrate the integrity of the segmented objects of LaGa. We also show multi-vies semantic
15

---

Tackling View-Dependent Semantics in 3D Language Gaussian Splatting
Table Impact of Kmax on segmentation performance (mloU).
Table 7. Effect of e on segmentation performance (mloU).
Kmax
5
10
15
20
30
0
0.1
0.2
0.3
0.4
mloU (%)
63.4
63.4
64.1
64.0
63.2
mloU (%)
62.6
63.0
62.1
64.0
60.6
segmentation results on 3D-OVS. Finally, we offer some representative scene-editing examples to highlight the practical significance of LaGa.
C.1. More Visual Comparisons with Existing Methods
Figure presents the results, which lead conclusions similar to those in Section 6.3. One phenomenon worth clarifying the needle-like borders around segmented objects. This effect arises from an inherent flaw of the 3D-GS representation, where 3D Gaussians trained fit multi-view RGB images without explicit object awareness. Consequently, numerous ambiguous Gaussians contribute the rendering of different objects. After segmentation, these Gaussians manifest as needle-like borders. Although certain 3D-GS segmentation methods (Cen et al., 2025b; Hu et al., 2024) have examined this issue, lies beyond the scope of this paper.
Text Prompts:
Text Prompts:
Text Prompts:
Text Prompts: "Knife" "Refrigerator"
"Pirate hut"
"Wavy noodles"
"Puper napkin"
"Poroelain hand"
"Apple"
Figurines
Ramen
Tentime
Waldo kitehen
LaGa (owrs)
LEGaussius
LangSplat
OpenCaussian
Figure 9, More qualitative comparison results on the LERF-OVS dataset.
C.2. Multi-view Segmentation Results the LERF-OVS Dataset
We present multi-view segmentation results for the LERF-OVS dataset in Figure 10. Both rendered 2D masks and the corresponding 3D objects (with backgrounds removed) are displayed. In the "Figurines-pink ice cream" and "Teeatime-coffee mug" examples, the 2D rendered masks are empty, yet the 3D objects become visible once occlusions are removed. This underscores a key difference between previous 2D-based approaches and our 3D method: because 2D methods rely on rendered feature maps, they cannot perceive objects occluded by obstacles. In contrast, conducting direct 3D understanding in 3D space free from this limitation.
16

---

Tackling View-Dependent Semantics in 3D Language Gaussian Splatting
C.3. Multi-view Semantic Segmentation Results the 3D-OVS Dataset
We also present multi-view semantic segmentation results on the 3D-OVS dataset, shown in Figure 11. Although LaGa is primarily designed for 3D space, it readily handles forward-facing scenes. The forward- facing nature of this dataset reduces the impact of view-dependent semanties. Nevertheless, LaGa achieves on-par performance with existing 2D-based methods.
C.4. Multi-granularity Segmentation Results on the LERF-OVS Dataset
As described in Appendix B, LaGa performs multi -ulti-granularity segmentation by training three levels of affinity features. We show this ability in Figure 12. The "Avg." setting denotes the default strategy that averages predictions across all levels. We observe that relying on single segmentation level is generally less reliable than the multi-level averaging approach. At the coarsest level, different objects may occasionally be merged due to under--egmentation by SAM. At the finest ("subpart") level, segeentgmnnttoonn often produces regions with limited semantic content, making CLIP-based classification less effective.
C.5. Scene Editing Results
We present a few representative scene-editing examples in Figure 13. By selecting 3D objects via text prompts, we can modify their appearance or remove them entirely from the scene. A more intriguing example involves transplanting a "cup" from the "Figurines" scene into the "Waldo-kitchen" scene. A more intriguing example involves transplanting a "cup" from the "Figurines" scene into the "Waldo-kitchen" scene. This demonstration not only confirms LaGa's ability retrieve high-quality objects but also highlights its potential for more complex interactive tasks, such as training Al agents by constructing simulation environments.
C.6. Mask Group Examples
We present visualization examples of mask groups obtained during the cross-view semanties grouping phase, shown in Figure 14. Masks representing the same 3D object are accurately grouped regardless of their relative size in the current view or the viewing angle, enabling a comprchensive aggregation of view-dependent semantics.
In addition to demonstrating the effectiveness of the semantics grouping approach, we aim to uncover the general causes of view-dependent semantics by analyzing the multi- viet masked images. We attribute view-dependent semantics primarily to key information loss under specific viewpoints. For instance, targets of interest may become occluded, fall outside the field of view, or appear too distant for their key features be recognized. These scenarios to the inherent limitations of the vision cone. Another contributing factor is poor data quality, illustrated by the spice jars in the 'Waldo Kitchen' scene. Blurred images in the collected dataset result in loss of detail for concrete objects. While the former issue is intrinsic, the latter can be mitigated through more careful data collection processes.
both common and unavoidable due
C.7. Examples of View-Depended Semantics
To further illustrate the phenomenon of view-dependent semantics, we present qualitative results in Figure 15. The relevance score for each 2D semantic feature is computed using Equation (10), where scores above 0.5 are classified as positive. The computed relevance scores are displayed above each figure, with the corresponding text prompts shown in the top-left corner of each row.
Notably, the semanties of the same 3D object exhibit significant variations across diffferent viewpoints. Even when the 2D masks are correctly assigned, the relevance scores for the same 3D object can range from 0.3 to 0.8. This variation strongly demonstrates the existence of significant view-dependent semantics.
17

---

Tackling View-Dependent Semantics in 3D Language Gaussian Splatting
Rendered Views
Rubics cube
Walde
Pink ice
Piguurines
Chopsticks
Ramon
Rendered Views
Sheep
Coffee mug
Plate
Rendered Views
Socket
Glass bottle
Bowl
Wildo Kilehean
Figure 10. Multi-view segmentation results on LERF-OVS dataset.
18

---

Tackling View-Dependent Semantics in 3D Language Gaussian Splatting
Bed
Red bag
Black leather shoe
Banana
Hand
Camera
White sheet
Bench
Portuguese egg tart
Orange cat
Green grape
Mini offload car
Dressing doll
Wood
Pebble concrete wall
Room
Shrilling chicken
Weaving basket
Rabbit
Dinosaur
Baseball
Wood wall
Sofa
Pikachu
stack of UNO cards
Xbox wireless controller
Gundam
Grey sofa
A red Nintendo Switch joy-con controller
Lawn
Red apple
New York Yankees cap
Stapler
Black headphone
Hand soap
Green lawn
Figure II. Multi-view semantic segmentation results the 3D-OVS dataset.
19

---

Tackling View-Dependent Semantics in 3D Language Gaussian Splatting
Figurines "Rubber duck"
Ramen: "Chopsriks"
Teatime: "Bag cookiies"
Wildo Kitchon: "Tonste"
Subpart
Whole
Avg.
Figure 12. Multi-granularity semantic segmentation results LERF-OVS dataset.
Original
Recolor
Object Removal
Manipullation
"Green Chair" Black
"Grezn Apple" Apple" Blue
Remove "Jake"
Remove "Mifly"
Bring the "cup" from "Figurines" "Kitchan"
Before
Afler
'11
Figure 13. Scene editing examples with the help LaGa.
20

---

Tackling View-Dependent Semantics in 3D Language Gaussian Splatting
Ramen
Figurines
Waldo Kitchen
Ramen
Figure Visualization mask groups obtained from the scene decomposition phase. Highlighted regions indicate the corresponding masks. Groups of masks are visually separated by dotted lines.
21

---

Tackling View-Dependent Semantics in 3D Language Gaussian Splatting
Cabinet: 0.47
0:48
0.49
0.60
0.62
TUp?
Keschup:
0.48
0.47
0.60
0.62
0.65
Pour-ever
vessel:
0.34
0.38
0.45
0.63
Table:
0.41
0.42
0.47
0.52
0.65
Twizzlers:
0.39
0.40
0.48
0.66
0.82
Milfy: 0.29
0.43
0.44
0.59
0.68
Figure View-dependent semantic variations within same mask group. Each row corresponds a different text prompt, shown the top-left corner. The relevance scores for individual 2D semantics are displayed ahove each figure.
22


Reviews:
[{"review_id": "HMeFrbdc9u", "reviewer": "Reviewer_ZzT9", "summary": "The paper introduces LaGa, a novel method designed to improve open-vocabulary 3D scene understanding by tackling view-dependent semantics in 3D Gaussian Splatting. LaGa works by breaking down the scene into 3D objects using multi-view 2D masks. It then aggregates the semantics of these objects through a combination of adaptive clustering and weighted relevance scoring. Experimental results highlight substantial advancements in 3D perception, with LaGa achieving an impressive +18.7% increase in mIoU (mean Intersection over Union) compared to earlier methods when tested on complex datasets.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank you for your time and effort dedicated to evaluating our work. We greatly appreciate your recognition of our motivation, methodological design, and the clarity of our writing. We find the remaining concerns mainly locates in the 'Experimental Designs Or Analyses' section. We hope our responses below will address them.\n\n>  (Experimental Designs Or Analyses) However, potential dataset bias, limited qualitative analysis, an untested recall threshol d (0.75), and narrow ablation scope (e.g., omitting decomposition details) are issues that could weaken generalizability and transparency if not addressed. \n\n**Potential dataset bias:**\nThank you for the comment. As you mentioned, our evaluation involves diverse benchmarks: LERF-OVS (complex 360° scenes), 3D-OVS (forward-facing views with diverse objects), and ScanNet (real-world scans). We believe this coverage is sufficient to demonstrate the effectiveness of our method. We are not entirely sure which specific bias is being referred to and would be happy to further discuss this concern.\n\n**Limited qualitative analysis:**\nPlease kindly refer to our appendix for more visualization results including more visual comparisons, multi-view visualizations, and results on the 3D-OVS dataset. To further strengthen our qualitative analysis, we will incorporate additional visualization results about multi-granularity segmentation cases and representative failure cases to the paper.\n\n**Untested recall threshold (0.75):** \nThe threshold of 0.75 was chosen based on analysis of the precision–recall trade-off under different cosine similarity values. Here, precision refers to the proportion of retrieved 3D Gaussians that actually belong to the corresponding 3D object. Low precision indicates that unrelated Gaussians are being retrieved for a given 2D mask.\n\nAs shown in the table below, lowering the similarity threshold (e.g., to 0.7) increases the proportion of samples with high recall (>0.9), but at the cost of significantly reduced precision. For example, at 0.7, the average precision of high-recall samples drops to just 13.4%.\n\nFrom these observations, we find that thresholds in the [0.75, 0.8] range strike a better balance. We conservatively select 0.75, where 50.2% of samples exhibit low recall, clearly demonstrating the challenge of view-dependent semantics. A threshold of 0.8 also yields valid results, with fewer high-recall samples but higher precision.\n\n|Threshold |0.5|0.6|0.7|0.75|0.8|0.9|\n|-|-|-|-|-|-|-|\n|Low / (Low+High) (%)|2.6|13.1|33.7|50.2|65.4|93.8|\n|Average Precision of High (%)|1.7|3.9|13.4|24.1|38.6|48.7|\n\nNote that the average precision does not reach 100%, as CLIP features operate at the semantic level, not the instance level. Therefore, 3D Gaussians with similar semantics but belonging to different objects may also be retrieved, even with a high threshold.\n\n\n**Narrow ablation scope:**\n\nThanks for the suggestion.\nTo further demonstrate the effectiveness of our design choices, we ablate two core components of the decomposition process:\n1. An ablation study on the **data resampling strategy** used for training the affinity features.\n2. A **hyperparameter analysis** of the HDBSCAN decomposition algorithm.\n\nEffect of Resampling Strategy:\n\n|              | F.   | T.   | R.   | W.   | Mean |\n|--------------|------|------|------|------|-------|\n| w/o Resampling | 54.8 | 70.5 | 40.8 | 57.5 | 55.9  |\n| w/ Resampling | 64.1 | 70.9 | 55.6 | 65.6 | 64.0  |\n\nAs shown above, removing the resampling strategy leads to consistent performance drops across all scenes, highlighting its importance for stable learning of affinity features.\n\nEffect of HDBSCAN Epsilon:\n\n| $\\epsilon$ | 0   | 0.1 | 0.2 | 0.3 | 0.4 |\n|--------------|-----|-----|-----|-----|-----|\n| mIoU (%)     | 62.6 | 63.0 | 62.1 | **64.0** | 60.6 |\n\nWe empirically set epsilon = 0.3 for all experiments. As the table shows, LaGa achieves strong performance within a broad and reasonable range ($\\epsilon \\in [0, 0.3]$). Performance degradation only occurs when $\\epsilon$ becomes too large, potentially causing unintended merging of semantically distinct objects. This analysis shows that LaGa is not sensitive to this hyperparameter.", "claims_and_evidence": "Yes.\nIt claims: The view-dependency of 3D semantics causes object understanding to vary with perspective, and simply projecting 2D semantics onto 3D Gaussians results in inaccurate and incomplete scene understanding due to this unaddressed variation.\nEvidence:  Figure3 and Figure2.", "methods_and_evaluation_criteria": "The evaluation uses LERF-OVS, 3D-OVS, and ScanNet datasets—covering complex 360° scenes, forward-facing views, and real-world scans—paired with mIoU and recall rate metrics to assess segmentation accuracy and object retrieval completeness. These choices are fitting because they test LaGa across diverse scenarios and directly measure its success in overcoming view-dependency, with supporting analyses and ablation studies reinforcing the method’s design and effectiveness for the problem at hand.", "theoretical_claims": "The paper lacks formal proofs for its theoretical claims—view-dependent semantics and the efficacy of scene decomposition with semantic aggregation—relying instead on logical reasoning and empirical evidence. I assessed their correctness through coherence, data support (e.g., semantic similarity and retrieval analyses showing view-dependency, and an 18.7% mIoU gain validating the method), and consistency, finding no issues. While not mathematically rigorous, the claims are well-supported by qualitative examples, quantitative results, and ablation studies, making them credible without formal proofs.", "experimental_designs_or_analyses": "I assessed the experimental designs and analyses validating the LaGa method for addressing view-dependent semantics in 3D Gaussian Splatting, focusing on datasets, metrics, ablation studies, and additional analyses. The datasets—LERF-OVS (complex 360° scenes), 3D-OVS (forward-facing, diverse objects), and ScanNet (real-world scans)—are diverse and relevant, testing LaGa across varied scenarios with clear preprocessing steps (e.g., SAM and CLIP usage). Metrics like mIoU and recall rate appropriately measure segmentation accuracy and object retrieval, while ablation studies confirm the value of adaptive clustering and weighting, with strong quantitative support (e.g., 10.6% mIoU drop without clustering).\n\nThe designs are mostly sound, with established benchmarks ensuring comparability and analyses like semantic similarity and retrieval integrity (e.g., 50% retrieval failure) reinforcing the problem’s scope. However, potential dataset bias, limited qualitative analysis, an untested recall threshold (0.75), and narrow ablation scope (e.g., omitting decomposition details) are issues that could weaken generalizability and transparency if not addressed. Overall, the experiments are robust but could benefit from broader testing and deeper analysis.", "relation_to_broader_scientific_literature": "The paper’s introduction of LaGa for open-vocabulary 3D scene understanding via 3D Gaussian Splatting significantly advances prior work like NeRF (Mildenhall et al., 2020) and CLIP (Radford et al., 2021) by explicitly tackling view-dependent semantics—a critical challenge rooted in multi-view stereo where object meaning shifts with perspective; LaGa innovates with 3D scene decomposition and view-aggregated representation, drawing from contrastive learning (NeRF-SOS, Fan et al., 2023), mask-based methods (Garfield, Kim et al., 2024), and clustering, offering a direct 3D solution validated on benchmarks like LERF-OVS and ScanNet, surpassing 2D-reliant approaches like LangSplat (Qin et al., 2024).", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "Figure 2 effectively illustrates the motivation behind the study, presenting a clear and intuitive example that guides readers seamlessly through the authors' reasoning.\nThe subsequent figures and experimental results strongly demonstrate the detrimental impact of this motivation on performance.", "other_comments_or_suggestions": "No.", "questions_for_authors": "No for now.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Wap7149Ovo", "reviewer": "Reviewer_dCLm", "summary": "This paper proposes LaGa, which explores open-vocabulary 3D Gaussian understanding by decomposing the 3D scene into objects and then establishing view-dependent semantic connections. The proposed approach does not rely on aligning 3D Gaussian semantic features with 2D semantic priors and therefore is simple but effective. Extensive experiments demonstrate the effectiveness of LaGa.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank you for instructive comments. We hope our response can help clear your concerns.\n\n## Weaknesses\n\n> W1: ... failure cases and analysis.\n\nThanks for the suggestion. We summarize key failure cases and provide representative examples:\n\n1. Bag-of-Words Effect in CLIP:\nWhen prompted with phrases like 'pirate hat on the rubber duck' or 'cookies in the plate,' CLIP often activates on individual nouns rather than the intended composite concept. This also affects LaGa. Future work may use large language models for better compositional grounding.\n\n2. Lack of Context in 2D Semantics:\nLaGa extracts 2D object-level semantics by feeding segmented regions into CLIP. While this reduces distraction from unrelated objects, it also removes necessary context. For instance, in Teatime, hooves are unrecognizable without the full sheep, but with the full object, only 'sheep' is detected since missing part-level cues.\n\nThese cases reflect the challenges of real-world open-vocabulary perception and the gap between model and human understanding. We will include this discussion and illustrative examples in the paper.\n\n> W2: ... understanding of the weighted descriptor relevance aggregation ... why works needs clarification ...\n\nWe hope the following clarification can help address your concern:\n\nSince LaGa does not manually control the segmentation (by SAM) or subsequent semantic feature extraction (by CLIP), the extracted 2D features may contain incorrect or noisy semantics. For example, the spine of a book may look like a 'knife' from certain views. To mitigate this issue, we adaptively weight each semantic descriptor based on two criteria:\n1. Directional Consistency:\nThis metric measures the cosine similarity between each individual descriptor and the global semantics of the object. Descriptors inconsistent with the global semantics receive lower weights. For instance, in the case of the book, descriptors representing a 'passport' align closely with the global semantics 'book,' resulting in higher weights, whereas descriptors resembling a 'knife' are suppressed due to semantic inconsistency.\n2. Internal Compactness:\nIn addition to clearly incorrect semantics, some descriptors result from ambiguous or noisy segmentation (e.g., SAM oversegmentation). They may lack clear meaning yet distort the global feature and compromise directional consistency. To address this, we introduce Internal Compactness, defined as the L2 norm of a descriptor. If the features in the cluster are semantically consistent and have similar directions, their average will have a relatively large L2 norm (close to 1). In contrast, if the features are inconsistent with diverse directions, their vector average will cancel out, resulting in a lower norm. Thus, the norm serves as a confidence measure for semantic reliability.\n \nTogether, these criteria help LaGa emphasize descriptors that are both globally consistent and semantically coherent. We will clarify this design further in the paper.\n\n> W3: typos.\n\nThanks. We will fix them.\n\n## Questions\n\n> Q1: (1) ... varying segmentation granularity across views? (2) ... inconsistent segmentations affect the final result? Further discussion\n\nThank you for the insightful suggestion.\n\n(1) To address varying segmentation granularity, we adopt a multi-level modeling strategy (Appendix B). For each 3D Gaussian, we learn three affinity features corresponding to SAM's subpart, part, and whole-level masks, and construct parallel view-aggregated representations. Predictions from all levels are averaged during inference.\n\n(2) When segmentation inconsistency occurs within a level, LaGa resolves it in a statistial learning manner: For a 3D region observed from multiple views with inconsistent segmentation, the dominant label (i.e., the granularity supported by the majority of views) is reinforced through training, enabling convergence toward a stable 3D prediction.\n\nDuring decomposition, in each level, all masks for the same 3D object are grouped together regardless of granularity. In the cross-view descriptor extraction, masks of different granularities will be assigned to separate semantic descriptors if they exhibit semantic discrepancies. This avoids semantic conflicts during aggregation.\n\nThis design also enables **multi-granularity semantic retrieval**: when queried with part-level prompts, the whole-level may exhibit a high response (e.g., 0.5) if it contains the semantic descriptor of that part. At the part level, the queried part will yield a high response (e.g., 0.6), while the remaining regions produce lower responses (e.g., 0). By averaging predictions, the model produces a high response (0.5 + 0.6) for the queried part, a moderately high response (0.5 + 0) for the whole object, and low responses (0 + 0) for unrelated objects. This hierarchical behavior aligns with human cognition and reflects real-world compositional semantics.\n\nWe will add the above discussion to our paper to help understanding.", "claims_and_evidence": "Yes. The superiority of the experimental results is demonstrated through numerical metric comparisons and visual comparisons. Ablation studies and appendix also provide demonstrations on the effectiveness of the proposed method.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "No proofs or theoretical claims.", "experimental_designs_or_analyses": "Yes. The authors compare their method with the latest approaches on widely used benchmarks. While there is a gap compared to the latest metrics on 3D-OVS dataset, LaGa achieves the best results on LERF-OVS and ScanNet datasets. The authors also conducted a thorough analysis of the experimental results.", "relation_to_broader_scientific_literature": "Compared to baseline methods, this paper's contribution lies in addressing view-dependent semantics and establishing cross-view semantic connections to explicitly capture view-dependent 3D semantics.", "essential_references_not_discussed": "None.", "other_strengths_and_weaknesses": "weaknesses:\n\n1. It would be better to display some failure cases and provide corresponding analysis.\n\n2. I did not find a reasonable understanding of the weighted descriptor relevance aggregation module. The ablation study of average pooling and max pooling does demonstrate the necessity of this module, but there doesn’t seem to be a clear explanation of why this design works. Some further clarification is needed.", "other_comments_or_suggestions": "typos:\n\nfirst paragraph of Section 4: “3D objects Section 5.3” -> “3D objects as illustrated in Section 5.3”", "questions_for_authors": "How do you handle the issue of varying segmentation granularity of SAM across different views? If there are inconsistent segmentations across different views, how does this issue affect the final result? Further discussion on this problem would be beneficial.\n\n\n\n######--------------\n\nUpdate: The rebuttal has addressed my concerns. I will maintain my score of weak accept.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "EU4qSNjcll", "reviewer": "Reviewer_QD2y", "summary": "The paper proposes to perform open-vocabulary semantic segmentation of 3DGS scenes that respect view-dependent and view-independent semantics. Using SAM masks, per-gaussian contrastive features are learned to learn 3D object clusters. The method is evaluated on LERF OVS and ScanNet datasets showcasing competitive quality.", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for careful evaluation.\n## Weaknesses\n> W1: Core part similar to SAGA, GARField ... seem crude and not impactful... \n\nThis is a misunderstanding about our method's core. Rather than decomposition, its core lies in the view-aggregated representation, which we believe is not crude.\n\nIt consists of two novel modules:\n1. Cross-view descriptor extraction adaptively captures multi-view semantics for objects.\n2. Weighted descriptor relevance aggregation refines the representation by assigning weights to descriptors. Unlike hard filtering with information loss, all descriptors are preserved.\n\nLaGa improves 18.7% mIoU over SOTA, and ablation shows +10.6% over baseline. This is clearly impactful.\n> W2: ... require careful tuning ...\n\nThe view-aggregated representation needs no sensitive tuning. Its K_max is fixed to 20 across all scenes/datasets. It is stable across K_max from 5–30.\n|K_max|5|10|15|20|30|\n|-|-|-|-|-|-|\n|mIoU|63.4|63.4|64.1|64.0|63.2|\n> W3: HDBSCAN ... how to obtain number of clusters N_p ... \n\nHDBSCAN automatically infers N_p. Its epsilon is fixed to 0.3 without per-scene tuning. As shown below, LaGa has stable performance across a wide range (0-0.3). Too large epsilons lead to object merging.\n|Epsilon|0|0.1|0.2|0.3|0.4|\n|-|-|-|-|-|-|\n|mIoU|62.6|63.0|62.1|64.0|60.6|\n> W4: Sensitive to SAM masks\n\nBoundary regions mis-segmented in one view often appear as interior in others. Through multi-view aggregation, these inconsistencies can be eliminated in 3D, similar to prior work (SemanticNeRF, SA3D).\n> W5: ... reason for k-means ... why not FPS\n\nWe adopt k-means to reduce noise by aggregating features locally in the semantic space. In contrast, FPS may preserve more outliers.\n\nFor the concern about averaging distant features, we add an denoising to discard features far from their centroids (brings a minimal +0.2% mIoU). The FPS performs worse:\n|FPS|Ours|Ours+Denoise|\n|-|-|-|\n|52.4|64.0|64.2|\n> W6: ... evaluations in 2D ...\n\nNo standard 3D benchmark exists for open-vocabulary 3D-GS. We think a sparse set of well-chosen 2D annotated views is a proxy, as neighboring views have redundant information. We check LERF-OVS annotations and find them sufficient in multi-view coverage. This is why most works use it as the main evaluation.\n> W7: Ablation\n> W7-1: ... -DW vs +DW on F. and T.? \n> W7-3: T., Avg pooling ... \n\nThe performance fluctuations in F. and T. (Table 4) stem from the sensitivity of fixed k-means clustering. Manually set K may be too small to capture semantic diversity or too large causing outliers. Both are harmful for hard cases.\n\nIn Figurines, querying \"Pikachu\" targets a plastic bag with a Pikachu print. Even K=5 splits the \"Pikachu bag\" semantics from \"bag\". By +DW, they are down-weighted as outliers. In Teatime, CLIP sometimes misclassifies a plate as \"apple\". Avg. pooling or small K merge such descriptors, while K=10 offers better separation. K=20 introduces additional noise. The avg. pooling performs well in Teatime because its objects have more consistent multi-view semantics than other scenes. Avg. pooling is independent of K, simply averaging all multi-view features per object.\n \nAlthough corner cases are rare, they affect performance (Pikachu: 97% to 31%; Apple: 87% to 21%) and bring instablity. Our adaptive strategy successfully handles them and enables consistent gains with +DW.\n> W7-2: Waldo kitchen...\n\nThanks for finding this. It is an error caused by swapped DW^c and DW^d results between the F. and W. columns. The correct results are\n||F.||W.|\n|-|-|-|-|\n|-DW|59.7||62.8|\n|+DW^c|61.6||59.2|\n|+DW^d|60.8||63.9|\n|+DW|64.1||65.6|\n> W7-4: ...adaptive scheme not clear winner\n\nThis is a misunderstanding. The adaptive scheme is designed to deliver consistently strong and robust performance across diverse scenes without per-scene manually tuning.\n> W8 (reference): SAGA, GARField ...\n\nThey are designed for class-agnostic segmentation, rather than our task. To our knowledge, no prior work evaluates them on LERF-OVS. For comparison, we adapt the released SAGA code for 3D-OVS to deliver the following results:\n\n|F.|T.|R.|W.|Mean|\n|-|-|-|-|-|\n|36.2|19.3|53.1|14.4|30.7|\n## Questions\n>Q1: Binarized gaussians with 2D GT masks.\n\nDuring evaluation, for each query, binarized gaussians are projected to 2D via 3D-GS rendering. The resulting 2D masks are compared with GT without any associating.\n>Q2: Cross-View Descriptor Extraction as ablation.\n\nDisabling it is infeasible, as 3D objects require at least one descriptor. Instead, we evaluate it by comparing adaptive scheme with alternatives (Sec. 6.4). '-DW' (Table 4) show results of this module work alone.\n>Q3: Larger clusters L2-norm > 1.\n\nThis won't happen since all features are L2-normlized before clustering (L. 192). \n>Q4: Train Langsplat?\n\nNo. We do not train LangSplat (neither CLIP features for Gaussians) in LaGa. L632-633 indicates we train three affinity features for each 3D Gaussian to maintain the multi-level SAM segmentation. We will clarify.", "claims_and_evidence": "The paper claims that prior methods for 3D semantic understanding suffer from view-dependency of 3D semantics which limits their quality  and robustness. The claim is supported by experiments on precision and recall rates for the 2D-3D semantic features.", "methods_and_evaluation_criteria": "The problem is well-motivated with experimental analyses. The core part of the method is learning contrastive features which is similar to SAGA, GARField among many others. The biggest drawback of this method are the steps that builds on top of it. Overall, the clustering and filtering approaches performed seem crude and not that impactful based on quantitative evaluations. \n\nThey also seem to require careful tuning of hyperparameters for it to work well. It would be great if the authors can comment on this.\n\nThe HDBSCAN performed on top of \\hat{f}_M seems to dictate the number of segmented objects in the scene and affect semantic view-aggregation downstream. But how is the number of clusters N_p obtained in the first place? A wrong value could under or over-segment the scene leading to misaligned semantics. \n\nSensitivity to SAM masks: How do you handle SAM masks that aren't consistent in mask boundaries for a particular object across multiple frames. This is very common artifact in the LERF dataset for \\textit{figurines}, \\textit{ramen}, and \\textit{teatime}.\n\nWhat is the reason for performing k-means clustering in Cross-View Descriptor Extraction? An alternative is to perform farthest point sampling that will represent the most \"diverse\" viewpoints. With kmeans there is a possibility of averaging out far away features which might be undesirable. \n\nThe quantitative evaluations are performed by rendering the 2D binary map. Its not clear if its completely sufficient to show 3D segmentation quality unless 1) evaluation is performed directly in 3D or 2) 360 degree 2D evaluation is performed. Only table 3 and figure 6 actually support the claim of 3D segmentation quality.\n\nTable 4: \n1) why is -DW better than +DW for fixed k=5,10,20 on figurines and teatime? \n2) For Waldo kitchen scene with adaptive clustering, -DW performs better than +DW^c and +DW^d. But the combined +DW is better than -DW. How is this possible? \n3) For Teatime scene, average pooling is better than fixed k=5,20 but worse than k=10. What is the reasoning for this? Is Avg pooling for k > 20?\n4) The adaptive scheme is not a clear winner in any one scene.", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "N/A", "relation_to_broader_scientific_literature": "The contrastive approach followed in the paper is in line with previous works like SAGA, GARField among others that perform 3DGS segmentation using 2D contrastive losses.", "essential_references_not_discussed": "- Table 1 SAGA, GARField are strong baselines to compare against which are missing in the evaluations.", "other_strengths_and_weaknesses": "The method follows a well-established approach to segment 3D Gaussians using contrastive losses. By performing clustering and further view-weighted associations on top of the contrastive features, they are robust to viewpoints and retain semantics for all gaussians belonging to a semantic entity. However this would be sensitive to hyperparameters and would require extensive tuning per scene/dataset.", "other_comments_or_suggestions": "- Eq. 3 should be performed per mask. M^I_{i} and not M^{I}\n- Eq. 8 should be C not C' for dimension of d_{i}.\n- Eq. 10  exp <d \\dot \\phi^{i}_{canon}>. \\dot -> ,", "questions_for_authors": "- For evaluation, how is the binarized gaussians associated with the 2D ground-truth masks? based on mask iou?\n- What is the effect of Cross-View Descriptor Extraction individually as an ablation?\n- For internal compactness, couldn't larger clusters have the L2-norm be much greater than 1?\n- L632-633: \"... follow LangSplat to train a three-layer model...\". Do you train a complete Langsplat model (learning clip features on gaussians)? why is it that necessary? It seems quite overkill for the purpose of getting multi-view consistent 2D segmentations.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "N/A"}}, {"review_id": "MypmYtE7Pc", "reviewer": "Reviewer_sKQC", "summary": "The paper addresses the challenge of view-dependent semantics in 3D Gaussian Splatting for language-driven open-vocabulary scene understanding. The authors propose LaGa (Language Gaussians), a method that decomposes 3D scenes into objects and constructs view-aggregated semantic representations by clustering and reweighting semantic descriptors based on multi-view semantics. Specifically, they weigh the descriptor relevance with the directional consistency and internal compactness. The paper claims significant improvements over state-of-the-art methods, particularly on the LERF-OVS dataset, with a +18.7% mIoU improvement.\n\n##Update after rebuttal\nI thank the authors for their clarification. I have no further concerns, but I remain neutral regarding this paper's novelty and significance. Therefore, I have decided to maintain my score as weak accept.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for your effort in reviewing our paper. We hope the following response would address your concerns.\n\n## Weaknesses:\n>W1: ... does not support multi-scale segmentation like LangSplat ...\n\nThank you for the comment. LaGa supports multi-scale segmentation, though not sufficiently emphasized in the main paper. Some details are provided in Appendix B under the implementation section. Specifically, during the scene decomposition, LaGa maintains three affinity features for each 3D Gaussian, which are supervised using SAM-extracted segmentation masks at the subpart, part, and whole levels, respectively. During inference, the final result is produced by averaging the responses from these three levels, enabling LaGa to capture semantic concepts across varying granularities.\n\nThis multi-scale capability is further reflected in the visualization results. For example, in Ramen scene shown in Fig. 6, OpenGaussian merges the 'kamaboko' with the surrounding noodles due to its lack of multi-scale segmentation ability, whereas LaGa successfully distinguishes it. Similarly, in Fig. 7, LaGa accurately segments the pirate hat from the rubber duck. Moreover, we verify that when queried with 'rubber duck', LaGa retrieves both the duck and its pirate hat, demonstrating its effectiveness in capturing hierarchical part-whole semantics.\n\nWe will incorporate the visual results of 'rubber duck' into the revised paper, along with more specifically designed visualizations to better illustrate the multi-scale segmentation ability of LaGa.\n\n>W2: ... engineering-driven and does not provide enough knowledge advancement. The idea of addressing the feature inconsistency has been proposed in OpenGaussian.\n\nThank you for the thoughtful comment. While our method includes practical design choices, we believe it also contributes meaningful insights to open-vocabulary 3D scene understanding. In particular:\n\n1. Identification and analysis of view-dependent semantics:\nWe are the first to systematically identify and quantitatively analyze the phenomenon of view-dependent semantics, as illustrated in Figures 3, 4, and 12. This issue, long overlooked by prior work including OpenGaussian, plays a critical role in open-vocabulary 3D understanding and poses unique challenges for multi-view semantic aggregation.\n2. An effective approach for semantic aggregation across views: \nWe propose an effective approach, LaGa, which is able to robustly suppress noisy or inconsistent semantics while preserving informative signals, without requiring manual rules or task-specific heuristics. We believe the simplicity, generalizability, and effectiveness of our approach can serve as a good foundation for future research in this area.\n\n**Comparison with OpenGaussian:**\n\nWhile both LaGa and OpenGaussian aim to address feature inconsistency, their motivations and methodologies differ fundamentally.\n\nOpenGaussian focuses on the **inconsistency between 3D point-level features and 2D pixel-level CLIP features**, attributing the problem to (1) inaccurate 2D–3D feature association caused by the alpha-blending in differentiable rendering, and (2) limited expressiveness due to 3D feature compression for rendering efficiency. To address this, OpenGaussian decomposes the 3D scene, so that it can adopt a hand-crafted strategy to directly assign a 2D CLIP feature from a specific view to all 3D Gaussians within an object. However, this hard assignment discards rich multi-view semantic cues, resulting in substantial information loss and ultimately sub-optimal performance.\n\nIn contrast, LaGa recognizes feature inconsistency as the **semantic discrepancy among multi-view observations of the same 3D object**, which arises naturally in 3D scene understanding. To address this, LaGa proposes to aggregate view-specific 2D semantics into comprehensive and robust 3D semantic representations through adaptive semantic descriptor extraction and descriptor reweighting. Note that LaGa does not involve any 3D vision-language feature training, and thus is also unaffected by the 2D–3D inconsistency problem that OpenGaussian aims to address. \n\nThe revelation of the critical view-dependent semantics issue and the corresponding improvements help LaGa achieve a significant performance gain: LaGa outperforms OpenGaussian by +18.7% mIoU under the same experimental setting, validating the effectiveness of our approach.\n\n## Questions:\n> Q1: Multi-scale segmentation?\n\nYes. Please refer to our response to W1.\n> Q2: Runtime statistics of this method (e.g. how long and how much memory does it take)?\n\nLaGa is highly efficient. We evaluate its runtime performance on the LERF-OVS dataset using a single NVIDIA RTX 3090 GPU. The inference time per query ranges from approximately 80 ms to 200 ms, with peak GPU memory usage between 5 GiB and 13 GiB. Note that compared with 2D methods like LangSplat, which takes about 250 ms per query on the rendered feature map, LaGa directly deliver 3D point-wise understanding.", "claims_and_evidence": "The claims made in the paper are generally supported by clear and convincing evidence. The authors provide extensive experiments to validate their motivation, including semantic similarity distribution analysis and semantic retrieval integrity analysis. The quantitative results on the LERF-OVS, 3D-OVS, and ScanNet datasets show improvements over existing methods, supporting the claim that LaGa achieves more comprehensive 3D scene understanding.", "methods_and_evaluation_criteria": "The proposed methods make sense for the problem at hand. LaGa's approach to decomposing 3D scenes into objects and constructing view-aggregated semantic representations is a logical way to address view-dependent semantics. The evaluation criteria are appropriate, with the authors using standard benchmarks (LERF-OVS, 3D-OVS, and ScanNet) to evaluate their method. The use of mIoU as a metric is standard in semantic segmentation tasks and provides a clear measure of performance.", "theoretical_claims": "The paper does not present any theoretical proof, so there are no theoretical claims to evaluate.", "experimental_designs_or_analyses": "The experimental designs and analyses are sound and well-executed. The authors conduct a thorough evaluation of their method on multiple datasets and provide both quantitative and qualitative results.", "relation_to_broader_scientific_literature": "This paper aims to resolve the inconsistency of the 2D language embeddings of CLIP, which is an important task as currently we only have 2D multimodal foundation models and can only obtain 3D features by distilling features from 2D models. Thus the 3D inconsistency of 2D models is an important problem to tackle.", "essential_references_not_discussed": "The references seem sufficient.", "other_strengths_and_weaknesses": "Strengths:\n1. The paper is well-motivated, aiming to address the inconsistency of the 2D CLIP embedding.\n2. The experimental results are strong, with improvements over state-of-the-art methods on multiple datasets.\n\nWeaknesses:\n1. The proposed method does not support multi-scale segmentation like LangSplat, which could hinder its practicality in open-vocabulary segmentation.\n2. Although the proposed method shows strong results, it is more engineering-driven and does not provide enough knowledge advancement. The idea of addressing the feature inconsistency has been proposed in previous work such as OpenGaussian.", "other_comments_or_suggestions": "No other comments.", "questions_for_authors": "1. Can the method perform multi-scale segmentation?\n2. What are the runtime statistics of this method (e.g. how long and how much memory does it take)?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "H9hIiNhdkt", "reviewer": "Reviewer_YHPs", "summary": "This paper proposes a novel method，named LaGA and it tackles the challenge of view-dependent semantics in language-driven open-vocabulary 3D scene understanding. \n LaGa decomposes the 3D scene into distinct objects and then builds view-aggregated semantic representations by clustering semantic descriptors and reweighting them based on multi-view information. Extensive experiments demonstrate that this approach effectively captures the key nuances of view-dependent semantics, leading to a more comprehensive understanding of 3D scenes.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank you for your constructive comments. We hope our response can help address your concern.\n\n## Weaknesses\n\n>W1: The claimed innovation in 3D Scene Decomposition shows notable similarities with the previous work on openGaussian. I recommend that the authors explicitly discuss these similarities and clarify how their approach differs from and improves upon the prior work.\n\nThanks for the insightful suggestion. We discuss the relationship between OpenGaussian and LaGa below:\n\n**Similarity:**\nBoth OpenGaussian and LaGa first decompose a 3D scene into objects before associating the scene with semantic information.\n\n**Key Differences in Motivation:**\nThe two methods are built upon fundamentally different motivations. OpenGaussian addresses the inconsistency between 3D point-level features and 2D pixel-level CLIP features, attributing it to \n(1) inaccurate 2D–3D associations introduced by alpha-blending during differentiable rendering, and \n(2) limited expressiveness from 3D feature compression for efficinet rendering. \nIn contrast, LaGa focuses on tackling the semantic discrepancy among multi-view observations of the same 3D object, which naturally arises in open-vocabulary 3D understanding. This shift in perspective leads to substantially different methodologies.\n\n**Key Differences in Methodology:**\nTo mitigate 2D–3D feature inconsistency, OpenGaussian decomposes the 3D scene so that it can directly assign a 2D CLIP feature to each Gaussian of an object. It employs a rule-based representative view selection strategy for this assignment. While straightforward, this strategy neglects rich information embedded in multi-view semantics and lead to suboptimal performance. \n\nIn contrast, LaGa extracts semantic descriptors across views using adaptive clustering and then adopts the weighted relevance aggregation to suppress noisy semantics and enhance robustness. This design enables LaGa to construct a more comprehensive and robust 3D semantic representation. Notably, LaGa does not involve any 3D vision-language feature training, and thus is also unaffected by the 2D–3D inconsistency problem that OpenGaussian aims to address. The design of LaGa enables it to tackle complex 3D objects with various multi-view semantics, evidenced by its significant +18.7% mIoU improvement over OpenGaussian.\n\n**On Scene Decomposition:**\nWe acknowledge that our scene decomposition strategy is not the core innovation of LaGa, which is inspired by prior contrastive-learning-based approaches. However, compared to OpenGaussian’s two-stage codebook-based decompsition pipeline, which requires manual tuning of two codebook sizes and thus incurs higher computational cost, LaGa adopts a more lightweight solution, i.e., using HDBSCAN to automatically determine the number of decomposed objects based on feature density. This results in comparable decomposition accuracy while significantly reducing system complexity.\n\nWe will incorporate this clarification and discussion in the revised paper for improved clarity and completeness.\n\n\n>W2: In addition to the quantitative comparisons for 3D open-vocabulary localization, I would also like to see 2D query results, as well as qualitative results on the mipnerf360 dataset— for example, for “Room” and “Garden”.\n\nThank you for the valuable suggestion. We provide several 2D segmentation masks in Figure 8, which demonstrate the 2D projections of LaGa’s 3D query results. Since LaGa does not involve any 2D CLIP feature learning, it is not intended for direct querying on the 2D image plane. Nevertheless, we believe that our 3D-centric paradigm can serve as a more flexible alternative to 2D methods, as the 3D query result can be rendered from arbitrary viewpoints without requiring repeated queries for each individual image.\n\nIn addition, we have conducted qualitative experiments on the MIP-360 dataset, including scenes such as Room and Garden, and observed that LaGa generalizes well to these scenes. The corresponding visualizations will be included in the revised paper.", "claims_and_evidence": "yes", "methods_and_evaluation_criteria": "yes, it does make sense.", "theoretical_claims": "Yes, I have reviewed the theoretical explanations provided in the paper for the following components: 3D Scene Decomposition, View-Aggregated Semantic Representation, and Weighted Descriptor Relevance Aggregation. I did not find any errors in these sections.", "experimental_designs_or_analyses": "Extensive experiments demonstrate that LaGa effectively captures key information from view-dependent semantics, enabling a more comprehensive understanding of 3D scenes.", "relation_to_broader_scientific_literature": "It is worth noting that the claimed innovation in 3D Scene Decomposition shows notable similarities with the previous work on openGaussian[1]. I recommend that the authors explicitly discuss these similarities and clarify how their approach differs from and improves upon the prior work. \n\n[1] OpenGaussian: Towards Point-Level 3D Gaussian-based Open Vocabulary Understanding", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "### Strengths\nThis paper is written clearly, with well-articulated motivation and clearly presented figures and charts. The experiments demonstrate that the proposed method can achieve precise open-vocabulary 3D Gaussian localization.\n### Weaknesses\nThe claimed innovation in 3D Scene Decomposition shows notable similarities with the previous work on openGaussian[1]. I recommend that the authors explicitly discuss these similarities and clarify how their approach differs from and improves upon the prior work.\n\n\n[1] OpenGaussian: Towards Point-Level 3D Gaussian-based Open Vocabulary Understanding", "other_comments_or_suggestions": "In addition to the quantitative comparisons for 3D open-vocabulary localization, I would also like to see 2D query results, as well as qualitative results on the mipnerf360 dataset— for example, for “Room” and “Garden”.", "questions_for_authors": "No", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The majority of reviewers leaned towards acceptance (scores 4, 3, 3, 3, 1), and the author rebuttal successfully addressed most concerns raised during the review process. One reviewer maintained a strong rejection, mentioning concerns regarding the method's generalizability, the potential need for hand-tuning in the k-means strategy, robustness across different scenarios, and the quantitative comparison with the SAGA baseline. While acknowledging these points were raised, the consensus of the remaining reviewers, the novelty of addressing view-dependent semantics, and the paper's improvements over prior SOTA support its acceptance into the conference. Therefore, I recommend accepting this paper for its contribution to 3D open-vocabulary understanding. 

The authors are encouraged to incorporate the feedback from all reviewers and, where possible, further clarify the points regarding tuning sensitivity and the comparison to SAGA in the final version to address the remaining concerns.
Decision: Accept (poster)