LangSplatV2: High-dimensional 3D Language Gaussian Splatting with 450+ FPS
Authors: Wanhua Li
Yujie Zhao
Minghan Qin
Yang Liu
Yuanhao Cai
Chuang Gan
Hanspeter Pfister
Venue: NeurIPS 2025 poster ()
Keywords: Gaussian Splatting
Real time
Language Field
Primary area: applications
TL;DR: We present LangSplatV2, which achieves high-dimensional feature splatting at 476.2 FPS and 3D open-vocabulary text querying at 384.6 FPS for high-resolution images.

Abstract: In this paper, we introduce LangSplatV2, which achieves high-dimensional feature
splatting at 476.2 FPS and 3D open-vocabulary text querying at 384.6 FPS for
high-resolution images, providing a 42 × speedup and a 47 × boost over LangSplat
respectively, along with improved query accuracy. LangSplat employs Gaussian
Splatting to embed 2D CLIP language features into 3D, significantly enhancing
speed and learning a precise 3D language field with SAM semantics. Such advancements in 3D language fields are crucial for applications that require language
interaction within complex scenes. However, LangSplat does not yet achieve real-
time performance (8.2 FPS), even with advanced A100 GPUs, severely limiting
its broader application. In this paper, we first conduct a detailed time analysis of
LangSplat, identifying the heavyweight decoder as the primary speed bottleneck.
Our solution, LangSplatV2 assumes that each Gaussian acts as a sparse code within
a global dictionary, leading to the learning of a 3D sparse coefficient field that
entirely eliminates the need for a heavyweight decoder. By leveraging this sparsity,
we further propose an efficient sparse coefficient splatting method with CUDA optimization, rendering high-dimensional feature maps at high quality while incurring
only the time cost of splatting an ultra-low-dimensional feature. Our experimental
results demonstrate that LangSplatV2 not only achieves better or competitive query
accuracy but is also significantly faster. Codes and demos are available at our project page: https://langsplat-v2.github.io.

LangSplatV2: High-dimensional 3D Language Gaussian Splatting with 450+ FPS
Wanhua Li
Yujie Zhao .2,,
Minghan Qin
Yang Liu
Yuanhao Cai*
Chuang Gan
Hanspeter Pfister*
Harvard University
University of Chinese Academy of Sciences
Tsinghua University
*Johns Hopkins University
MMIT-IBM Watson Al Lab
UMass Amherst
Abstract
In this paper, introduce LangSplatV2, which achieves high-dimensional feature splatting at 476.2 FPS and 3D upen-vocabulary text querying at 384.6 FPS for high-resolution images, providing a 42 speedup and 47 x boost over LangSplat respectively, along with improved query accuracy. LangSplat employs Gaussian Splatting to embed 2D CLIP language features into 3D, significantly enhancing speed and learning a precise 3D language field with SAM semantics. Such ad- vancements in 3D language fields crucial for applications that require language interaction within complex scenes. However, LangSplat does not yet achieve real- time inference performance (8.2 FPS), even with advanced A A100 GPUs, severely limiting its broader application. In this paper, we first conduct a detailed time analysis of LangSplat, identifying the heavyweight decoder as the primary speed bottleneck. Our solution, LangSplat assumes that each Gaussian acts as sparse code within a global dictionary, leading the learning of a 3D sparse coefficient field that entirely eliminates the need for a heavyweight decoder. By leveraging this sparsity, we further propose an efficient sparse coefficient splatting method with CUDA optimization, rendering high- ligh--imnnsional feature maps at high quality while incurring only the time cost of splatting an ultra-low-dimensional feature. Our experimental results demonstrate that LangSplatV2 not only achieves better or competitive query accuracy but is also significantly faster. Codes and demos are v2. github.io io
available project page:
Introduction
Seamless and intuitive interactions between humans and complex 3D environments 11 2 are paramount for a wide range of applications, such as augmented reality
and intelligent robotics 5 7 To achieve such interactions, systems must understand and respond to natural language queries in real time. This capability hinges on advancements in 3D language fields tech- nology at the intersection of vision-language models [8] 9 and 3D environmental modeling [10 [T].
LangSplat
pioneering model in this domain, leverages 3D Gaussian Splatting to embed 2D CLIP language features into 3D spaces. This method has significantly accelerated the 3D query time, achieving speeds up to 199 x faster than its predecessors [13,, which is critical for applications in scenarios where rapid response is essential. Although many recent improvements have been proposed [4] the inference speed remains a major concern, especially at high resolutions, which hinders their widespread application.
Real-time querying crucial for applications such real-time navigation [6], interactive gaming (16]. and on-the-fly educational tools [17], where delays can disrupt user experience and functionality. To
Equal contribution.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

identify the speed bottleneck of LangSplat, we decompose the entire querying process into three stages: rendering, decoding, and post-procsssing. Then, we provide a detailed time analysis on the LERF dataset with one A100 GPU. Results in Table [ show that each query takes 122.1 ms for LangSplar. With some simple engineering modifications, the rendering and post-processing time can be significantly reduced, we term this new version of LangSplat as LangSplat*. The performance of LangSplat* revealed that the decoding stage, which is responsible for transforming low-dimensional latent features into high-dimensional CLIP features with a heavy- multi-layer perceptron (MLP), is the primary bottleneck. LangSplat introduces the MLP decoder to significantly reduce the training memory and time cost. Consequently, an additional decoding stage with an MLP is inevitable for test-time querying and reducing the dimensionality of high-dimensional features lowers the accuracy of the queries. However, simply removing the decoder and directly training the high- dimensional CLIP feature for each Gaussian dramatically increases the rendering time. As shown in Figure as the feature splatting dimension increases, the rendering time of LangSplat significantly increases. Specifically, rendering a feature with a dimension of 1536 (assuming we render three semantic levels of 51 2--dimmesiooaa CLIP feature fields in parallef) is 15 times slower than rendering a 3-dimensional field on one A100 GPU.
To address the decoding bottleneck in LangSplat, we introduce LangSplatV2, which drastically enhances querying speed without sacrificing querying accuracy. As shown in Figure [I] our LangSplatV2 successfully decouples ren- dering speed from the dimensionality of ren- dering features, enabling the rendering of high- dimensional features at the computational cost of splatting an ultra-low-dimensional feature. As the millions of 3D Gaussian points actu- ally represent limited number of unique se- mantics for 3D scene, our LangSplat as- sumes that each Gaussian can be represented as a sparse coding over a group of global ba- sis vectors. Then, we derive that learning high-dimensional feature field is equivalent to learning 3D coefficient field and global code- book. For each 3D Gaussian point, instead of augmenting a language feature, we learn sparse coefficients. During testing, we utilize the spar- sity and propose an efficient sparse coefficient splatting method with CUDA optimization, which effectively decouples the rendering dimension with feature dimensions. As our method removes the autoencoder and directly learns the 3D field from high- -obhdmmennioonll 2D CLIP features, more accurate language fields are also obtained. Experimental results demonstrate that our LangSplatV2 not only achieves higher accuracy but also delivers a substantial speedup: 42 faster than SpngSplat for sigh-dimensional feature rendering and 47 x faster for open-vocabulary 3D querying.
35
Lang
LangSp a/V2 on
30
LangSpaat
LangSp u012 on RTX4090
25
Langgp R7X3300
La Sg wor2 on RIX3090
Banderting Time
25
20-
25. 2024 2048
Festure Dimensioo
Figure 1: Feature rendering time comparison with different GPUs. Note that the less advanced GPUs (RTX 3090 and RTX 4090) cannot accommodate the LangSplat model with feature dimensions of 1024 or higher due to running out of memory.
2 Related Work
3D Gaussian Splatting. Recently, 3D Gaussian Splatting (3D-GS) 11] has received huge atten-
tion [18] 19 due to its speed advantage (2), 21 over neural radiance fields (NeRFs) [T0] As a
fundamental 3D modeling technology, it's been widely used in many downstream tasks 12 23 24
25 26 27 28, (29]. Yang et al. (30] extended 3D Gaussian Splatting to dynamic scenes by modeling deformable 3D Gaussians. Concurrently, 4D Gaussian Splatting [3I] proposes to use multi-resolution voxel planes to model deformed Gaussians and attains high-quality video rendering results in real- time. Due to the explicit 3D modeling nature of 3D Gaussian Splatting, has also been used for 3D editing. Gaussianeditor [32) offers swift and controllable 3D editing by tracing the editing target throughout the training process. Another concurrent work (33 edits 3D scenes using text instruction, which attains two times faster training speed compared with Instruct-NeRFNNRRF 34 Many methods [5 36 37 utilize 3D Gaussian Splatting for text-driven 3D generation. GSGEN [3 incor-
porates direct geometric priors from 3D Gaussians and obtains a text-to-3D generation model that

---

captures -egu-freueency components. 3D-GS has also been adopted for generative 3D reconstruction, which has recently attracted significant interest owing its effectiveness with sparse observations.
The first framework of its kind was introduced by Liu et al. [3 who pioneered generative 3D) reconstruction by modeling as a temporal generation task. Wang et al. 40] subsequently accelerated this approach by using one-step diffusion. Recently, 3D language fields have also embraced 3D-GS. LangSplat [12] adopts 3D-GS to model 3D language field and is 199 x faster than LERF [13]. However, LangSplat is still not a real-time method, which undermines its broad application prospects.
3D Language Field. The concept of 3D language fields [4 intersection of that can be manipulated and queried using natural language. LERF [3] explores 3D open-vocabulane
has emerged as an models and 3D modeling, aiming to create interactive environments
text queries by embedding the CLIP feature into a 3D field. Liu et al. 47 also distilling knowledge
from pre-trained foundation models like CLIIP 8 and DINO 48 into NeRF [Q] It further proposes distribution alignment and nature-distributinn alignment losses to address the CLIP feature ambiguity issue. LangSplat improves LERF with 3D Gaussian Splatting and attains a significant employing 3D Gaussian Splatting for 3D language field modeling, they usually suffer from imprecise language fields like LERF, which has been well addressed by LangSplat. Following LangSplat, GOI [52] proposes simplifying feature selection by dividing the feature space with a hyperplane, keeping only the features most relevant to the query. GAGS [15] enhances multiview consistency by linking SAM's prompt point density with camera distances and introducing an unsupervised granularity factor to selectively distill consistent 2D features. However, these methods both use decoder to map the oow-dimensional feature to high-dimensional space, which is a primary bottleneck of real-time query as shown section 3.2
speedup. There are also some other works 49 50 51
3D Gaussian Splatting Acceleration. To reduce computational overhead and improve render- ing speed, recent works have proposed various strategies to accelerate 3D-GS. One line of work compresses the Gaussian representation using vector quantization and codebooks. For instance, Compressed 3D Gaussian Splatting (53] adopts sensitivity-aware clustering and quantization-awan training to compress both directional colors and Gaussian parameters, achieving up to 31 x com- pression and 4x rendering speedup. CompGS [54] similarly applies ammanns-bssed quantization to achieve over 40x storage reduction with minimal quality loss. Other approaches focus on struc- tural regularization or architectural simplification. Self-Organizing Gaussian Grids (55] reorganize Gaussian parameters into spatially coherent 2D grids, reducing redundancy via local homogeneity. LightGaussian (56) uses pruning based on global significance and vector quantization, achieving 15x compression and 200+ FPS rendering. C3DGS (57) proposes to reduce the number of Gaussians via a learnable mask and introduces compact view-dependent color encoding, achieving a 25x reduction in storage. Techniques like Speedy-Splat (58) and SORT-Free Splatting [59] improve runtime performance through optimized rasterization and differentiable approximations of alpha blending. In contrast to these works that focus on accelerating RGB-based scene rendering, our method tackles the unique challenge of handling high -igh-dimensional Gaussian features. Extending prior compression techniques (53) [66] to our setting would require learning and quantizing such high-dimensional attributes per Gaussian, leading to excessive memory usage. Moreover, directly rendering with full 512D features remains computationally demanding. contrast, our method circumvents the need to directly learn and render high- dimenssial features for 3D Gaussian points.
3 Proposed Approach
3.1 Preliminaries
3D Gaussian Splatting employs boint-based rendering technologies 660 61 and models the scene geometry as set of 3D Gaussian points, Each Gaussian point is associated with the following attributes: Gaussian center position E IR3, covariance matrix Σ E R3x3, color described e e R by spherical harmonic (SH) coefficients, and opacity o E R.. Each Gaussian point is represented as:
((p)= 1
n) T - =)))
The covariance matrix Σ is further represented with rotation matrix and scaling factor matrix:
Σ = RSS RT,
where R E I4 denotes the rotation matrix and S E R² represents the scaling matrix.
3

---

Table I: The stage-wise time cost (ms) for LangSplat and our improvements on the LERF dataset with one A100 GPU. LangSplat* is modified from LangSplat with simple engineering optimization. LangSplatV2 achieves a speed of up to 384.6 FPS for open-vocabulay 3D querying.
Method
Rendering
Decoding
Post-Processing
Total
Speed (FPS)
LangSplat
6.0
83.1
33.0
122.1
8.2
LangSplat*
2.0
83.1
0.5
85.6
11.7
LangSplatV2
2.0
0.1
0.5
2.6
384.6
Based on EWA volume splatting [62]. 3D Gaussian Splatting projects the 3D Gaussian points onto a
2D image plane, which blends the ordered Gaussian points that overlap with the rendered pixel v:
C(v) = Σ Go EE
a1)))) ==1
(3)
where N represents the ordered Gaussian points within a tile, and a; o The G.P means the projected 2D Gaussian distribution of the i-th 3D Gaussian point.
As 3D Gaussian Splatting exhibits excellent real-time rendering speed while maintaining high rendering quality even at 1080p resolution, many methods [31 30 631 have adopted as a 3D scene modeling technology to accelerate the rendering speed. LangSplat aims to build a 3D language field to support open-vocabulary querying within 3D spaces. extends each 3D Gaussian with language embedding and supervises the 3D language Gaussians with 2D CLIP image features. SpnaSplat presents two main contributions to make it faster and more accurate. First, instead of using multiple patch-wise CLIP features with varying scales, which leads to imprecise and vague 3D language fields, LangSplat employs the CLIP features of SAM masks with three pre-defined SAM hierarchical semantic scales to obtain precise language fields with clear boundaries. Second, CLIP embeddings are high-dimensional, directly splatting at CLIP feature space poses huge challenges in training memory and time cost. Therefore, LangSplat proposes a scene- scene-specific autoencoder, which compresses the high- nigh-dimensional (512-D) CLIP features into low-dimensional (3-D) latent features. The rendering process of LangSplat follows:
F()) = Eia
-0)=Emmn VEN
fas II 1=1
(1-aa).
where fi E dRdd represents the augmented d-dimensional latent feature at i-th 3D Gaussian point and F(v) denotes the rendered latent feature at pixel v. After obtaining the rendered feature, LangSplat uses the decoder (a(F())) ) IR" 2 from the trained autoencoder g to decode the latent feature back to CLIP feature space. In the end, the LangSplat attains an accurate 3D language field while being 199 x faster than the previous state-of-the art method LERF [33.
3.2 Bottleneck Analysis
While LangSplat achieved significant speedup over other methods, still cannot perform real-time open-vocabon vocn-vocab 3D querying at high resolution. However, there is high demand for real-time 3D querying for many applications, such as Augmented Reality (AR) (64, intelligent robotics [3].
To further improve the query speed of LangSplat, we first analyze each step of the query process and identify the bottleneck. For given text query, the inference of LangSplat can be divided into three stages: rendering, decoding, and post-processing. The rendering stage will render the learned 31) language Gaussians into 2D image plane and get a rendered latent feature map E # xd where H, W denote the height and width of the 2D image size, respectively. As the rendered feature map only encodes language features in the low-dimensional latent space, decoding stage followed to obtain the feature map at CLIP space with decoder 9a( (F) E H #xxx WXD where the decoder (.(-)) implemented with an MLP. The last post-processing stage computes the relevancy score with the obtained D-dimensional feature following LERF [13] and remove some noise in the relevancy score map. Specifically, a mean filter is applied to the relevancy score map. Note that LangSplat adopts the three semantic levels introduced by SAM 665] so the above operations are repeated three times for three SAM semantic levels and the post-processing stage needs select one semantic level for prediction with some specific strategies [2]

---

Global Codebook
512-dim Language Feature
Ground Truth
Multi- View Image
Cac
Top
Hiererdéed Semurtii
Nart.
Reatering
CLIP
Gaussiass
Sparse Cuefficient
L-dim Coefficient Map
Preprecess
Figure 2: The framework of LangSplatV2. LangSplatV2 introduces sparse coefficient for each Gaussian point and a shared global codebook for the entire scene.
We test the stage-wise inference time of LpngSplat on the LERF dataset with one A100 GPU. The results in Table[]]shww that the rendering stage takes 6.0 ms, the decoding stage takes 83.1 1 ms, and the post- post-processing stage takes 33.0 ms. While all stages occupy a considerable amount of time, we can significantly improve them through two simple modifications. First, we notice that LangSplat implements the pro--proessiing stage on the CPU, which could be slow when it involves mean filter operations. We implement the post-processing stage on the GPU and observe that the time cost of the post-processing stage is now negligible compared with other stages. The second modification is scale parallelization. LangSplat performs text querying on three semantic levels sequentially, meaning the rendering stage is performed three times separately. However, we can perform inference at three semantic scales parallel. For example, instead of splatting d-dimensional features three times, we directly splat 3dd-dimenioonal features for once. We use LangSplat* to denote the version with these simple modifications. We observe that for LangSplat*, the rendering stage takes 2.0 ms, and the post-proe post-processing stage takes 0.5 ms. In these three stages, the decoding phase occupies 97. % of the total query time, making it the primary bottleneck in achieving real-time 3D querying.
3.3 3D Sparse Coefficient Field
Different from 3D Gaussian Splatting, which renders 3-dimensional RGB color, 3D language fields need to splat nigh-dimensional features. Directly splatting high-dimensional features significantly decreases the rendering speed, as shown in Figure [ To address this issue, existing methods usually model d-dimensional (d << D) latent features in 3D followed by either an online decoder [5T) or an offline decoder [12] to decode latent features back to D-dimensional features in 2D image space. The high dimension gap between latent features and decoded features implies heavyweight MLP is required to ensure the accuracy of the decoding stage, which becomes the primary speed bottleneck.
In V2ng instead of splatting the d-dimensional latent language features, propose to model a 3D sparse coefficient field. As shown in Eq. LangSplat assigns each Gaussian point with language feature f., which represents the semantics associated with the Gaussian point. As LplaSplat could create millions of Gaussian points, there will be millions of unique language features. However, this is highly inefficient the unique semanties within scene quite limited and much smaller than the number of Gaussian points. As a matter of fact, many Gaussian points share similar semantics. Therefore, we assume the language embedding of every Gaussian point within scene can be represented as a sparse coding of L global basis vectors S = [[1, $2,..., st) E gLxD These L basis vectors serve as the global codebook and each Gaussian point computed by linearly combining a small number of local basis vectors. We assume that only K (K << L) basis vectors from global codebook are used to represent the language embedding of a Gaussian point. Then we define w; [u,,11, Wi,2 .... 1 E R x as the associated sparse coefficients for i-th Gaussian point, where Σt= Wi,1 1 and only K elements are non-zero values while all other elements are zeros. With slightly abuse of the notion f,, the language embedding fr E RDD of i-th Gaussian point is represented as:
f. f- f.=w, W.S Σ 11
(5)
5

---

If we directly render the D-dimensional language field without compressing CLIP features, we have:
S
Σ
w.
So
II (1,))
(6)
IEN
j=1
where S is the rendered D-dimensional CLIP feature. We e₁ 11,111100 az), then have:
S
=
w,Se; e,w,)S.
iEN
iEN
Eq. shows that rendering the D-dimensional CLIP features is equivalent first rendering the sparse coefficients (1),, and then performing a matrix multiplication with the global dictionary S.
Therefore, we propose learn L-dimensional sparse coefficient for each Gaussian point and global codebook with a size of L, basis vectors. To learn sparse distribution for the coefficient wi, first apply softmax function normalize the L-dimensional parameter, then preserve the top-K values as non-zeros elements and set the remaining elements to zeros. We will re-normalize the top-K values with their sum ensure the of the coefficient equals one. The T--dimensional 3D) sparse coefficient field and the L basis vectors are jointly learned. Figure 2 visualizes the framework.
Compared to LangSplat, Lang requires only simple matrix multiplication after rendering the weight map, instead of relying on a heavyweight decoder, thus overcoming LangSplat's s main inference speed bottleneck. Furthermore, the D D-dimensional global codebook eliminates reconstruction loss from dimensionality reduction, enabling better modeling of high-dimensional features the CLIP space and improving query accuracy.
3.4 Efficient Sparse Coefficient Splatting
By learning 3D sparse coefficient field, the MLP decoder is entirely removed, eliminating the associated computational overhead of the MLP. However, we still need to perform 3L-dimensional feature rendering and matrix multiplication. Our experiments show that the time for matrix multiplication (in Table T listed as the decoding stage of LangSplatV2) is negligible compared to the rendering process. However, rendering nigh-dimensional features with dimensionality L remains computationally demanding. To address this issue, we propose an efficient sparse coefficient splatting method with CUDA optimization. By exploiting the sparsity of the coefficient field, we can achieve L-dimensional feature rendering at the cost of only K --dimensional rendering, where K < L.
In the CUDA implementation of 3D Gaussian Splatting and LangSplat, each thread performs alpha-blending of the [V] ordered Gaussian points within a tile. With an L-dimensional rendering, each thread sequentially computes L channels, leading to a computational complex- ity of ZLW When becomes sufficiently large, this alpha-blending computation becomes the key bottleneck and significantly increases the overall computational overhead as L grows, as shown in Figure
3xtx512
Dinoonsoe
Level
Gaursaa Rostierr
Level
To mitigate this, we utilize the sparse nature of the learned coefficient field during test-time querying, as shown in Figure 3 Although each Gaussian point has an L-dimensional coefficient, only K dimensions contain non-zero values. Therefore we can only perform beladbblending for the non-zero elements, as the blending of zero elements does not alter the result. This effectively reduces the computational complexity from O(L L) to ()) with significantly smaller than L. In practice, we set K 4, which allows to simultaneously render three semantic scales with an effective feature rendering dimension CUDA of only 12, yielding a high-quality feature map of 1, 536 dimensions. It makes the rendering highly efficient without compromising feature quality.
Dinsonsien
Divvcnstion
Figure 3: Our efficient sparse coefficient splatting method accelerates the speed of fiiha-blending by utilizing the property of the learned sparse coeffi- cient field and neglecting zero elements.

---

Table 2: Quantitative comparisons of open-vocabulary 3D object localization and 3D semantic segmentation on the LERF dataset. We report the mean accuracy (%) and the mean IoU scores (%).
3D Object Localization
3D
Semantic Segmentation
Method
Ramen Kitchme Figtuieeniiginnnssvvve Laverlllammn Teatime KitchenFigurinss
GS-Grouping 667
32.4
69.5
50.0
44.6
49.1
26.4
54.0
31.3
34.6
36.6
LEGaussian 49
69.0
79.7
63.6
57.1
67.4
20.2
32.3
22.3
23.4
24.6
GOI 152
56.3
67.8
68.2
44.6
59.2
33.7
55.8
54.5
23.9
42.0
GAGS 16
69.0
88.1
90.9
78.6
81.7
46.8
60.3
55.8
53.6
54.1
LangSplat
12
73.2
88.1
95.5
80.4
84.3
51.2
65.1
44.5
44.7
51.4
LangSplatV2
74.7
93.2
86.4
82.1
84.1
51.8
72.2
59.1
56.4
59.9
Specifically, each Gaussian's L-dimensional sparse cocfficient w, can be fully represented by its top- K non-zero elements. We store these as two K K-diimensional arrays for each Gaussian: top- indices and top-K coefficients. The top-K indices the positions of the top-K K non-zero elements within the I-dimensional vector and the top-K coefficients are the values of these top-K non-zero elements. During rendering, each CUDA thread performs alpha-blending only for the K non-zero coefficients. For each Gaussian point within the tile, the CUDA thread will access the indices and coefficients of the top-K elements and perform weighted summation solely on these K-dimensional indices, avoiding computation on zero elements. In this way. LangSplatV2 can obtain high-dimensional (D)
feature splatting results the cost of only
(K) feature splatting,
4 Experiments
4.1 Datasets and Details
Datasets. We evaluate our method on the LERF, 3D-OVS, and Mip-NeRF360 datasets. The LERF dataset [13]. captured using the iPhone App Polycam, contains in-the-wild scenes. For the open- vocabulary 3D object localization task, we adopt the augmented localization annotations provided by LangSplat [2 on the LERF dataset. Additionally, we use the segmentation ground truth from LangSplat 12 for the open- open-vocabulary 3D segmentation task on LERE Beyond LERF, we also conduct 3D segmentation experiments on the 3D-OVS and Mip-NeRF360 [66] datasets. The 3D-OVS dataset 47 includes collection of long-tail objects captured in diverse poses and backgrounds. Moreover, we evaluate our method on Mip-NeRF360, which consists of multi-view indoor and outdoor scene images, with segmentation labels annotated by GAGS [[5. For evaluation, we use localization accuracy for the 3D object localization task and report the average IoU scores for the open-vocabulary 3D segmentation task.
Implementation Details. Following LangSplat 12 we use the OpenCLIP ViT-B/16 model to extract CLIP features. We employ the VIT-H model for SAM (65] segment images and obtain masks with three hierarchical semantics. The codebook size I. is set to 64 and the K is set to 4. During test-time querying, we render three semantic scales simultaneously. leading to the actual rendering dimension of 12. The 3D Gaussians are first trained with RGB supervision for 30,000 iterations reconstruct the RGB scene. Then we train another 10,000 iterations for the 3D sparse coefficient field by fixing all other 3D Gaussian parameters. All our experiments are conducted on one A 100 GPU.
4.2 Quantitative Results
Time Analysis. To assess the speed improvement of LangSplatV2 over its LangSplat, we conducted detailed time analysis using the LERF dataset, with the computations performed on single 1000 GPU. Table T presents a stage-wise breakdown of the time costs for LangSplat, LangSplat", and LangSplat LangSplat, our initial model, achieves a frame rate of 8.2 FPS. The LangSplat* improvements mainly focused on rendering and post-processing optimizations, which reduced the total time to 85.6 ms and increased the speed to 11.7 FPS. Here, the rendering time was significantly cut to 2.0 ms, and the post-processing time was reduced to less than 1.0 ms with our proposed simple engineering modifications. contrast, LangSplatV2 proposed an efficient sparse coefficient splatting method, that entirely removed the MLP decoder in the decoding stage. The only operation in the
7

---

LanpSplat
LangSpar122
AGB
LangSpper
LangSpbes2
por
"hoove""
Figure 4: Qualitative comparisons of open- vocabulary 3D object localization on the LERF dataset. The red points are the model predictions and the black dashed bounding boxes denote the annotations. We observe that LangSplatV2 generates better results than LangSplat.
decoding stage is performing matrix multiplication between the rendered L-dimensional coefficient and the global dictionary, which takes only 0.1 ms. While completely removing the MLP decoder, our LangSpaitV2> only needs 2.0 ms render the sparse coefficient, leading to overall 476.2 FPS for obtaining high-dimensional feature maps. In the end, LangSplatV2 achieved an exceptional reduction in total querying time to 2.6 ms. These results not only demonstrate a substantial enhancement in processing speed but also affirm the real-time capabilities of LangSplat for high-resolution, complex 3D scene querying.
Main Results on the LERF dataset. Table 2 demonstrates the compar- isons on the LERF dataset. For the open-vocabulary 3D object localiza- tion task, LangSplatV2 achieved the highest accuracy with notable im- provements over previous methods for most scenes, including "ramen", "figurines", and "teatime". In the "Kitchen" scene, LangSplat achieved higher accuracy than LangSplat This difference mainly due to the limited sample size (22 examples in total), with only two-prediction gap (21 vs. 19). Given LangSplat's high variance in this scene, we ran it multiple times, obtaining an average 17.5 + 1.8 correct predictions, which suggests that LangSplat provides more consistent performance advantage across different scenarios. Results on 3D semantic segmentation demonstrate that LangSplatV2 outperforms other methods, and it consistently surpasses LangSplat across all scenes, highlighting the effectiveness of our proposed method.
Table 3: Quantitative 3D semantic segmentation results on 3D-OVS, reported as mean IoU (%)
Method
Bed Bench Room Sofa Lawn Overall
FFD I
56.6
6.1
25.1
3.7
42.9
26.9
LERF 13
73.5
53.2
46.6
27.0
73.7
54.8
3D-OVS 47
89.5
89.3
92.8
74.0
88.2
86.8
GS Grouping 67
83.0
91.5
85.9
87.3
90.6
87.7
LEGaussian 49
84.9
91.1
86.0
87.8
92.5
88.5
GOI [52]
89.4
92.8
91.3
85.6
94.1
90.6
LangSplat 12
92.5
94.2
94.1
90.0
96.1
93.4
LangSplatV2
93.0
94.9
96.1
92.3
96.6
94.6
Main Results on the 3D-OVS dataset. Table 3 shows the quantita- tive comparisons of open-vocabulary 3D semantic segmentation perfor- mance, on the 3D-OVS dataset across five test scenes. We see that LangSplatV2 achieves the highest overall mean IoU score of 94.6%. significantly outperforms LangSplat across all different scenes, further val- idating its ability to build more aceu- rate language fields.
Table 4: Quantitative 3D semantic segmentation results Mip-NeRF360, reported as mean IoU (%).
Method
Room Counter Garden Bonsai Overall
GS- Grouping 67
54.4
47.7
40.4
54.1
49.2
LEGaussian 49
25.5
35.3
33.2
22.3
29.1
GOI 52
60.3
46.6
59.8
67.3
58.5
GAGS 15
65.2
61.1
61.2
70.5
64.5
LangSplat
53.2
68.8
51.9
55.4
57.3
LangSplatV2
64.3
75.1
65.0
73.1
69.4
Main Results on the Mip-NeRF360 dataset. Table 4 presents the comparison results on the Mip- NeRF360 dataset. Notably, LangSp1atV2 achieved a significant improvement in the overall average IoU score, reaching 69.4%, which is marked advancement over LangSplat's 57.3%.

---

Table 5: Ablation study on codebook size L. We Table 6: Ablation study on different K. We report report the average performance the 3D object the average performance of the 3D object local- localization and 3D semantic segmentation tasks the LERF dataset.
ization and 3D semantic segmentation tasks on
on the LERF dataset.
L
32
64
128
K
2
6
Accuracy (%)
72.8
84.1
84.1
Accuracy (%)
79.4
84.1
84.4
83.6 59.9
IoU (%)
53.9
59.9
60.5
loU (%)
54.4
59.9
59.9
LERF: Ramen
3D-OVS Sofa
Mip-NeRF360: Bonsal
RGB
Ground Truth
RGS
Ground Truth
RGB
Ground Truth
LangSplat
LangSgletV2
LangSplet
LangSplutV2
LangSplat
Lang5platV2
chopatidss
Alachu
Gundow
controllyr
LERF: Kitchen
3D-OVS: Bed
Mip- Hip-NeRF330:: Room
Ground Truth
RGB
Ground Touth
RGB
Ground Truth
LangSplatV22
LangSplatVV2
LangS
LangSplatt22
#1
Ansie
ketchup
contera
white
speaber
HGl
wooden Aoor
Figure 5: Qualitative comparisons of open-vocabulary 3D semantic segmentation on the LERF, Mip-NeRF360 and 3D-OVS dataset. We can that LangSplatV2 generates better masks than LangSplat, which shows the effectiveness of our LangSplatV2.
Ablation Study. Table 5 we evaluate the effect of codebook sizes L on the localization and segmentation tasks. We report the mean accuracy and loU scores on the LERF dataset. Our results reveal that increasing from 32 to yields notable improvement in both localization accuracy (72.8% to 84. 1%) and segmentation loU (53.9% to 59.9%). This suggests that larger codebook size better captures the semantic fields in complex scenes, allowing for more precise language representations. However, further increasing results in only slightly increased performance, suggesting that L = 64 has already reached saturation on the LERF dataset. Table further reports the effects of different K. We observe that K 4 saturate the performance of the Gaussian model, and larger will increase the rendering dimensions, thereby slowing down the rendering speed. More ablation studies and details can be found in the supplementary material.
Training Cost. Table 7 shows comparison of training cost on the LERF [13 dataset with one 100 GPU. Compared with LEGaussians [9 and LangSplat [12) LangSplatV2 comes with increased training cost due to the need construct high-dimensiinal semantic fields during training. Although our LangSplatV2 incurs higher training costs compared to LangSplat and LEGaussians, primary focus this paper is on improving the model's test-time performance for deployment. As demonstrated in the experiments, LangSplatV2 achieves significantly higher inference speed-47xx faster than LangSplat and 14x faster than LEGaussians on the LERF dataset- under comparable memory consumption, while also yielding higher segmentation accuracy. Furthermore, compared with naive LpnaSplat (directly trained with 512 dimensions). LangSplatV2 more efficient with the proposed global codebook and sparse coefficient field.
Discussion. Recent work has also explored codebook- based language field representations, such as LEGaussians 49). However, approach departs significantly from theirs in several key aspects.

---

Table 7: Quantitative comparisons of training cost on the LERF dataset.
Method
LangSplat 12
LEGaussian
499
LangSplatV2
LangSplat-51122 12
Training Time (h)
1.0
1.3
3.0
45.0
Training Memory (GB)
6.2
11
21.2
47.4
Table Quantitative comparisons with LEGaussian on three benchmarks.
Segmentation IoU (%)
Query Time (ms)
GPU Memory (GB)
Method
LERF 3DOVS
Mip.
LERF 3DOVS Mip.
LERF
3DOVS Mip.
LEGaussian 49
24.6
88.5
29.1
36.7
59.6
58.0
8.2
10.5
14.0
LangSplatV2
59.9
94.6
69.4
2.6
4.8
3.8
7.2
11.6
16.9
First, LEGaussians construct 2D feature codebook from 2D images while LangSplatV2 directly learns global codebook 3D space, which is more efficient. Second, LEGaussians still rely on an MLP to compress features and do not exploit sparsity. In contrast, our method completely removes the MLP and further enforces and leverages sparsity in both representation and rendering. As shown in Table 8 method achieves superior semantic segmentation accuracy, faster inference speed, and comparable memory usage across three benchmark datasets, which demonstrates the effectiveness of approach. For more detailed comparison and discussion, please refer to the Appendix.
4.3 Qualitative Results
Figure 4 visualizes the open- vocabulary 3D object localization results. We observe that LangSpaltV2 can give more accurate localization predictions compared with LangSplat. For example, LangSplatV2 can give the correct prediction for "pumpkin" while LangSplat entirely fails in this query. 5 visualizes the opeablocab 3D segmentation results in three datasets. We generate
observe that LangSplatV2 example, in the "Kitchen" scene, LangSplat predicts noisy mask for the "ketchup" query, while our LangSplatV2 generates more precise and clean masks.
accurate masks compared with LangSplat. For
Conclusion
Conclusion. In this paper, we have presented LangSplatV2 for tigh-dimensional language feature splatting in 3D ooen-vocyyulary querying, designed to overcome the speed limitations of LangSplat. Our analysis identified the decoding stage as the primary bottleneck in LangSplat. By modeling each Gaussian point as sparse code over a global dictionary, LangSplatV2 removes the need for a heavyweight decoder. We further proposed an efficient sparse coefficient splatting method with CUDA optimization, allowing LangSplatV2 achieve nigh-dimensional feature splatting results at the cost of splatting ultra-low- low-dimensional features.
Limitations and broader impacts. While LangSplatV2 demonstrates improved performance and faster inference compared to LangSplat, comes with increased training cost due to the need to construct nigh-dimensional semantic fields during training. Furthermore, inherits the semantic representations from the CLIP model, it also inherits its inherent biases. Ad- dressing such biases remains open research challenge and may benefit from recent advances in fairness-aware versions of CLIP.
method directly
Acknowledgment
This research is supported in part by the NIH grant R01HD104969.
References
[1] Sosuke Kobayashi, Eicchi Matsumoto, and Vincent Sitzmann. Decomposing nerf for editing via feature field distillation. Advances in Neural Information Processing Systems, 35:23311-293 -233300,, 2022.
10

---

[2] Vadim Tschernezki, Iro Laina, Diane Larlus, and Andrea Vedaldi. Neural feature fusion fields: 3d distillation of self-supervised 2d image representations. In 2022 International Conference 3D Vision (3DV), pages 443-453, IEEE, 2022.
[3] Alan Craig. Understanding augmented reality: Concepts and applications. 2013.
[4] Ronald Azuma. survey of augmented reality. Presence: teleoperators & virtual environments, 6(4):355-385, 1997.
[5] Songyou Peng, Kyle Genova, Chiyu Jiang, Andrea Tagliasacchi, Marc Pollefeys, Thomas Funkhouser, et al. Openscene: 3d scene understanding with open vocabularies. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 815 824, 2023.
[6] Chenguang Huang, Oier Mees, Andy Zeng, and Wolfram Burgard. Visual language maps robot navigation. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 10608- 10615. IEEE, 2023.
[7] Paola simulated environments for visual question answering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5056-5066, 2022.
Hui Wu, Letao Wang. Rogerio S Feris, and Vicente Ordonez. Simvqa: Exploring
[8] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748-8763, PMLR. 2021.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36, 2024.
[10] Ben Mildenhall, Pratul Srinivasan, Matthew Tancik, Jonathan Barron, Ravi Ramamoonti,, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. Commanccations of the ACM, 65(1):99-106 2021.
[11] Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics, 42(4):1-14, 2023.
[12] Minghan Qin, Wanhua Li, Jiawei Zhou, Haoqian Wang, and Hanspeter Pfister. Langsplat: 3d language gaussian splatting. CVPR, 2024.
[13] Justin Kerr, Chung Min Kim, Ken Goldberg, Angjoo Kanazawa, and Matthew Tancik. Lerf: Language embedded radiance fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 19729 19739, 2023.
[14] Ola Shorinwa. Jiankai Sun, and Mac Schwager. Fast-splat: Fast, ambiguity-free semantics transfer in gaussian splatting. arXiv preprint arXiv:2411.13733.22 2024.
[15] Yuning Peng. Haiping Wang, Yuan Liu, Chenglu Wen, Zhen Dong, Bisheng Yang. Gags: Granularity- aware 3d feature distillation for gaussian splatting, arXiv preprini 1rXiv:2412.13652 2024.
[16] Raymond Koon Chuan Koh, Henry Been-Lirn Duh, and Jian Gu. An integrated design flow in user interface and interaction for enhancing mobile ar gaming experiences. In 2010 IEFE International Symposium on Mixed and Augmented Reality-Arts, Media, and Humanities, pages 47-52. IEEE, 2010.
[17] Robert Godwin-Jonse Jones. Augmented reality and language leamingg: From annotated vocabulary to place- based mobile games. 2016.
[18] Zehao Yu, Anpei Chen, Binbin Huang. Torsten Sattler, and Andreas Geiger. Mip-splatting: Alias-fre 3d gaussian splatting. In Proceedings of the IEEF/CVE conference on computer vision and pattern recognition, pages 19447-19456, 2024.
[19] Yang Fu, Sifei Liu, Amey Kulkarni, Jan Kautz, Alexei A Efros, and Xiaolong Wang. Colmap- free 3d gaussian splatting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20796-2088 20805, 2024.
[20] Jixuan Fan, Wanhua Li, Yifei Han, Tianru Dai, and Yansong Tang. Momentum- gs: Momentum gaussian dif-listillation for high-quality large scene reconstruction. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages 25250-2500 25260. 2025.

---

[21] Haonan Han. Rui Yang. Huan Liao, Jiankai Xing, Zunnan Xu, Xiaoming Yu, Junwei Zha, Xiu Li, and Wanhua Li. Reparo: Compositional 3d assets generation with differentiable 3d layout aligmment. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 25367-25377, 2025.
[22] Heng Yu, Joel Julin, Zoltán Á Milacski. Koichiro Ninuuma, and László Jeni. Cogs: Controllable gaussian splatting. arXiv preprint arXiv:2312.05664 2023.
[23] Xian Liu, Xiaohang Zhan, Jiaxiang Tang, Ying Shan, Gang Zeng. Dahua Lin, Xihui Liu, and Ziwei Liu. Hu- mangaussian: Text-driven 3d human generation with gaussian splatting, arXiv preprint arXiv:2311.17611, 2023.
[24] Zhicheng Lu, Xiang Guo, Le Hui, Tianrui Chen, Min Yang, Xiao Tang, Feng Zhu, and Yuchao Dai. 3d geometry-aware deformable gaussian splatting for dynamic view synthesis. arXiv preprint MiXiv:24040022777 2024.
[25] Kangjie Chen, Yingji Zhong, Zhihao Li, Jiaqi Lin, Youyu Chen, Minghan Qin, and Haoqian Wang. Quanti- fying and alleviating co- adaptation in sparse-view 3d gaussian splatting. arXiv preprint arXiv:2508. 12720. 2025.
[26] Yuanhao Cai, Zihao Xiao, Yixun Liang, Minghan Qin, Yulun Zhang, Xiaokang Yang, Yaoyao Liu, and Alan I. Yuille. Hdr-gs: Efficient high dynamic range novel view synthesis at 1000x speed via gaussian splatting. Advances Neural Information Processing Systems, 37:68453-68871, 2024.
[27] Dongbin Zhang, Yunfei Liu, Lijian Lin, Ye Zhu, Yang Li, Minghan Qin. Yu Li, and Haoqian Wang. Guava: Generalizable upper body 3d gaussian avalar. arXiv preprint arXiv:2505.0333 2025.
[28] Yang Liu, Xiang Huang, Minghan Qin. Qinwei Lin, and Haoqian Wang. Animatable 3d gaussian: Fast and high-quality reconstruction of multiple human avatars. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 1120-1129, 2024.
[29] Donghin Zhang, Chuming Wang. Weitao Wang, Peihao Li, Minghan Qin, and Haoqian Wang. Gaussian in the wild: 3d gaussian splatting for unconstrained image collections. In European Conference on Computer Vision, pages 341 441-959. Springer, 2024.
[30] Ziyi Yang, Xinyu Gao, Wen Zhou, Shaohui Jiao, Yuqing Zhang, and Xiaogang Jin. Deformable 3d gaussians for high- -ideliyyyity monocular dynamic scene reconstruction. arXiv preprint arXiv:2309.1311 13101, 2023.
[31] Guanjun Wu, Taoran Yi, Jiemin Fang, Lingxi Xie, Xiaopeng Zhang, Wei Wei, Wenyu Liu, Qi Tian, and Xinggang Wang. 4d gaussian splatting for real-time dynamic scene rendering. In CVPR, 2024.
[32] Yiwen Chen, Zilong Chen, Chi Zhang, Feng Wang, Xiaofeng Yang, Yikai Wang. Zhongang Cai, Lei Yang, Huaping Liu, and Guosheng Lin. Gaussianeditor: Swift and controllable editing with gaussian splatting. arXiv preprint vrXiv:2311.145211 2022.
[33] Jiemin Fang, Junjie Wang, Xiaopeng Zhang, Lingxi Xie, and Qi Tian. Gaussianeditor: Editing gaussians delicately with text instructions. arXiv preprint arXiv:2311.1603 2023.
[34] Ayaan Haque, Matthew Tancik, Alexei Efros, Aleksander Holynski. and Angjoo Kanazawa. Instruct- nerf2nerf: Editing 3d scenes with instructions. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pages 19740-19750, 2023.
[35] Shijie Zhou, Zhiwen Fan, Dejia Xu, Haoran Chang, Pradyumna Chari, Tejas Bharadwaj, Suya You, Zhangyang Wang, and Achuta Kadambi. Dreamscene360: Unconstrained text-to-3d scene generation with panoramic gaussian splatting. arXiv preprint 4rXiv:2404.06903, 2024.
[36] Jiaxiang Tang, Zhaoxi Chen, Xiaokang Chen, Tengfei Wang, Gang Zeng, and Ziwei Liu. Lgm: Large multi-view gaussian model for high-resolution 3d content creation. arXiv preprint wwXiv:2402.0505.0 2024.
[37] Jiawei Ren, Liang Pan, Jiaxiang Tang, Chi Zhang, Ang Cao, Gang Zeng, and Ziwei Liu. Drcamgaussian4d: Generative 4d gaussian splatting. arXiv preprint 174442232..77142, 2023.
[38] Zilong Chen, Feng Wang. and Huaping Liu. Text-to-3d using gaussian splatting. arXiv preprint 11X5852309.6688. 2023.
[39] Fangfu Liu, Wenqiang Sun, Hanyang Wang, Yikai Wang. Haowen Sun, Junliang Ye, Jun Zhang, and Yueqi Duan. Reconx: Reconstruct any scene from sparse views with video diffusion model. arXiv preprint arXiv:2408.166 16767, 2024.
12

---

[40] Hanyang Wang. Fangfu Liu, Jiawei Chi, and Yueqi Duan. Videoscene: Distilling video diffusion model to generate 3d scenes in step. In 2025 IEEE/CVF Conference on Compuder Vision and Patiern Recognition (CVPR), pages 16475-16485. IEEE, 2025.
[41] William Shen, Ge Yang, Alan Yu, Jansen Wong, Leslie Pack Kaelbling, and Phillip Isola. Distilled feature fields enable few-shot language-guided manipulation. arXiv preprint arXiv:2308.07931, 2023.
[42] Yawar Siddiqui, Lorenzo Porzi, Samuel Rota Buló, Norman Müller, Mathhia Nießner, Angela Dai, and Peter Kontschieder. Panoptic lifting for 3d scene understanding with neural fields. In Proceedings of the 2023.
IEEE/CVF Conference on Computer Vision and Puttern Recognition. pages
[43] Shuaifeng Zhi, Tristan Laidlow, Stefan Leutenegger, and Andrew Davison. In-place seene labelling and understanding with implicit scene representation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15838-15847, 2021.
[44] Hao Li, Roy Qin, Zhengyu Zou, Diqi He, Bohan Li, Bingquan Dai, Dingewn Zhang, and Junwei Han. Lang- surf: Language-embeddee surface gaussians for scene understanding. arXiv preprint arXiv:2412.1666 2024.
[45] Kangjie Chen, BingQuan Dai, Minghan Qin. Dongbin Zhang, Peihao Li, Yingshuang Zou, and Haoqian Wang. Sigaussian: Fast language gaussian splatting in sparse views. arXiv preprint arXiv:2412.083.0 2024.
[46] Wanhua Li, Renping Zhou, Jiawei Zhou, Yingwei Song, Johannes Herter, Minghan Qin, Gao Huang, and Hanspeter Pfister. 4d langsplat: 4d language gaussian splatting via multimodal large language models. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 22001 22001-22011, 2025.
[47] Kunhao Liu, Fangneng Zhan, Jiahui Zhang, Muyu Xu, Yingchen Yu, Abdulmotaleb El Saddik, Christian Theobalt, Eric Xing, and Shijian Lu. Weakly supervised 3d open-vocabulary segmentation. Advances in Neural Information Processing Systems, 111553333-34455 2023.
[48] Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF international conference computer vision, pages 9650-9660, 2021.
[49] Jin Chuan Shi. Miao Wang, Hao-Bin Bin Duan, and Cha Guan. Language embedded 3d gaussians for open- vecnbooaayl scene understanding. In Proceedings of the IEFF/CVF Conference Computer Vision and Pottern Recognition, pages 5333-5343, 2024.
[50] Xingxing Zuo, Pouya Samangouei, Yunwen Zhou, Yan Di, and Mingyang Li. Fmgs: Foundation model embedded 3d gaussian splatting for holistic 3d scene understanding. arXiv preprint arXiv:2401.01970. 2024.
[51] Shijic Zhou, Haoran Chang, Sicheng Jiang, Zhiwen Fan, Zehao Zhu, Dejia Xu, Pradyumna Chari, Suya You, Zhangyang Wang, and Achuta Kadambi. Feature 3dgs: Supercharging 3d gaussian splatting to enable distilled feature fields. arXiv preprint ar Xxi::2312.032000 2023.
[52] Yansong Qu, Shaohui Dai, Xinyang Li, Jianghang Lin, Liujuan Cao, Shengehuan Zhang, and Rongrong Ji. Goi: Find 3d gaussians interest with an optimizable open- apen-vocabllary semantic-space hyperplane, 2024.
[53] Simon Niedermayr, Josef Stumpfegger. and Rüdiger Westermann. Compressed 3d gaussian splatting for accelerated novel view synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10349-10358, 2024.
[54] KL Navaneet, Kossar Pourahmadi Meibodi, Soroush Abbasi Koohpayegani, and Hamed Pirsiavash. Compgs: Smaller and faster gaussian splatting with vector quantization. European Conference on Computer Vision, pages 330-349. Springer, 2024.
[55] Wieland Morgenstern, Florian Barthel, Anna Hilsmann, and Peter Eisert. Compact 3d scene representation via self-organizing gaussian grids. In European Conference on Computer Vision, pages 18 34. Springer, 2024.
[56] Zhiwen Fan, Kevin Wang, Kairun Wen, Zehao Zhu, Dejia Xu, Zhangyang Wang, et al. Lightgaussian: Unhounded 3d gaussian compression with 1,5x reduction and 200+ fps. Advances in neural information processing systems, 37:140138-1 140155, 2024.
[57] Joo Chan Lee, Daniel Rho, Xiangyu Sun, Jong Hwan Ko, and Eunbyung Park. Compact 3d gaussian representation for radiance field. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21719 21729,, 2024.
13

---

[58] Alex Hanson. Allen Tu, Geng Lin, Vasu Singla, Matthias Zwicker, and Tom Goldstein. Speedy-splat: Fast 3d gaussian splatting with sparse pixels and sparse primitives. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 21537-21546, 2025.
[59] Qiqi Hou, Randall Rauwendaal, Zifeng Li, Hoang Le. Farzad Farhadzadeh, Fatih Porikli, Alexei Bourd, and Amir Said. Sort-free gaussian splatting via weighted sum rendering. arXiv preprint arXiv:2410.1899 2024.
[60] Hanspeter Pfister, Matthias Zwicker, Jeroen Van Baar, Markus Gross. Surfels: Surface elements as rendering primitives. In Proceedings of the 27th annual conference on Computer graphics and interactive techniques, pages 335 342, 2000.
[61] Matthias Zwicker, Hanspeter Pfister, Jeroen Van Baar, and Markus Gross. Surface splatting. In Proceedings of the 28th anmual conference on Computer graphics and interactive techniques, pages 371-378, 2001.
[62] Matthias Zwicker, Hanspeter Plister, Jeroen Van Baar, and Markus Gross. Ewa volume splatting. In Proceedings Visualization, 2001. 01. pages 29-538. IEEE, 2001.
[63] Zi-Xin Zou, Zhipeng Yu, Yuan-Chen Guo, Yangguang Li, Ding Liang, Yan-Pei Cao, and Song-Hai Zhang. Triplane meets gaussian splatting: Fast and generalizable single-view 3d reconstruction with transformers. arXiv preprint arXiv:2312.09144 2023.
2023.
[64] Priyanka Jain, Nivedita Bhirud, Subhash Tatale, Abhishek Kale, Mayank Bhale, Aakanksha Hajare, and NK Jain. Real-time interactive ar for cognitive learning. In AI, IoT, Big Data and Cloud Computing for Industry 4.0, pages 219-239. Springer, 2023.
[65] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead. Alexander C Berg, Wan- Yen Lo. et al. Segment anything. In Proceedings of IEEF/CVF International Conference on Computer Vision, pages 4015-4026, 2023.
[66] Jonathan T. Barron, Ben Mildenhall, Dor Verbin, Pratul Srinivasan, and Peter Hedman. Mip-nerf 360: Unbounded anti-aliased neural radiance fields. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5460-5469, 2022.
[67] Mingqiao Ye, Martin Danelljan, Fisher Yu, and Lei Ke. Gaussian grouping: Segment and edit anything in 3d scenes. In European Conference on Computer Vision, pages 162-179. Springer, 2024.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The main claims made in the abstract and introduction accurately reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer:
[Yes]
Justification:
We discussed the limitations in the Conclusion Section.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
15

---

Justification:
This paper does not involve theoretical contributions.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We have included an Algorithm to demonstrate how to reproduce our method. We have released the code.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
16

---

Answer: [Yes]
Justification:
We have released the source codes.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cm
coiioooooolyllli for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ ccpppp.ccppulli//guuii wwwiiicgguie///eeeeeeeeic CoodeSubeiiieeee ssi wswonnolli onnolicy)))) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We have included it in the Experiments Section.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [NA]
Justification: It is not included in all previous work in this field.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
17

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We have included it in the Experiments Section.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: /tte:::///uuis..eeeee futiinggttgss//ppppps..ccc////te
Answer: [Yes]
Justification: Our research conducted in the paper conforms the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We have discussed the broader impact in the Conclusion Section.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to
18

---

generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative socictal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: This paper poses such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
We have properly cited the original paper.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, wwppppswppteeeee whworee.wthhhe....c. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's is creators.
13. New assets
Question: Are new assets introduced in the paper well documented and the documentation provided alongside the assets?
19

---

Answer: [NA]
Justification:
This paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
20

---

The core method development in this research does not involve LLMs as any important, original, or non-standard components.
Justification:
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non- standad components.
Please refer to our LLM policy hattps://neurips... 200p://ee////oofeeeeee////////// for what should or should not be described.
21

---

A Algorithm
The proposed efficient sparse coefficient splatting process is shown in Algorithm !
Algorithm Efficient Sparse Coefficient Splatting
Initialize W E R to zero
for Gaussian point i E N do Retrieve top-K indices [f1. ..., jk( } and corresponding coefficients (Wijh Wi.z! Wiijx for each j E {i1, .... jk do
W += waj a4 I (1 am)
end for end for
B Discussion
LangSplatV2 proposes to model the high-dimensional language features of each Gaussian point as sparse code within global dictionary. While similar concepts, such as quantizing Gaussian point attributes (e.g., SH coefficients [56). opacity [53), and semantic features [49] into codebook, have been explored in the domains of compression [53] and language embedding [49], our method is fundamentally differrntt
In the compression domain, existing approaches 53 56 first learn unique attributes for each Gaussian point, which are then quantized into codebook using a quantizer. Extending this concept to our scenario would require learning 122dimmesiooall semantic features for each Gaussian point before quantization. This process significantly increases memory consumption and training time, often exceeding the capacity of GPUs like the NVIDIA 3090 or 4090, leading to out-of-memory (OOM) erros. Moreover, even with advanced GPUs that can handle the training. the rendering process would still involve managing 512-dimensional features, As shown in Figure I this dramatically reduces rendering speed compared to V2
In the context of language embedding, LEGaussians 49 adopt codebook-based approach to accelerate training. However, our method is significantly different from LEGaussians in three key aspects: 1) Global 3D Codebook vs. 2D Codebook: LEGaussians first train 2D codebook by quantizing features in 2D images, then learn 3D model to predict the one-hot class category indicating the index of the codebook. contrast, LangSplatV2 learns global 3D codebook shared among all 3D Gaussian points. By avoiding the additional reconstruction errors caused by first quantizing features the 2D image plane, our approach reduces overall reconstruction error and more efficiently utilizes the codebook in 3D space. 2) Eliminating the MLP Bottleneck: LEGaussians rely on an MLP to project each Gaussian point's ' features into a one-hot class category representing the index of the codebook. Our method removes the need for an MLP entirely, thereby eliminating the speed bottleneck and improving computational efficiency. 3) Efficient Sparse Coefficient Splatting: LangSplatV2 introduces Efficient Sparse Coefficient Splatting, which reduces the effective rendering dimensions to K. In contrast, LEGaussians renders with the full feature dimensions of Gaussian points, which inherently slows down the rendering process.
Furthermore, due to the reliance on an MLP decoder, LEGaussians still operate in lower-dimensional feature space, which limits the expressiveness of its learned representations. Additionally, its two- stage 2D codebook approach fails to effectively incorporate 3D priors, resulting in a suboptimal learned field. In contrast, LangSplat V2 directly models high-dimensional features within global 3D codebook, effectively capturing the underlying 3D structures. As evidenced by the results in Table our approach outperforms LEGaussians across multiple benchmarks while being significantly faster, demonstrating superior scene reconstruction quality and more accurate language-grone grounded 3D representations.
To conclude, LangSplatt2 demonstrates a significant leap forward in the efficiency and scalability of modeling high-dimensional language features in 3D scenes. By leveraging the proposed 3D sparse coefficient fields and efficient sparse coefficient splatting techniques, our method reduces computational overhead while maintaining high fidelity in 3D language scene rendering.
22

---

BGB
LangSpiat
LangSplam22
LangSpaz
LangSptar12
"ledl Iearn"
"ley oup
BGB
LangSpat
LangSplatt2
RGB
LungSplar
LangSptatt2
Tnakder dloc with bwayy"
Figure 6: More qualitative comparisons of open-vocabulary 3D object localization the LERF and Mip-NeRF360 datasets. The red points are the model predictions and the black dashed bounding boxes denote the annotations. We observe that LangSplatV2 generates better results than LangSplat.
Las Solae.
Oveendll Tuch
AGR
Langsppltt
Long Bergoorrr222
Oround Trefh
oale"
pou"
Lngöpperz
Groand
Whenpe
lang
Lanefplt
Larpbrert2
Shesing Aee"
Jonkhle canhiae GANONA Rod"
Voage
10
"eessiig Andir"
Wrthyy cheinr
Figure 7: More qualitative comparisons of open- apen-vocabulary 3D semantic segmentation on the LERF, Mip-NeRF360 and 3D-OVS dataset. We can see that our LangSplatV2 generates better masks than LpngSplat, which shows the effectiveness of our LangSplatV2.
C Evaluation Details
After calculating the similarity between the rendered semantic features and the CLIP features of the query text, we obtained relevance maps M at different semantic levels. Then, we use different strategies for different datasets to obtain the corresponding query results.
LERF. For object localization, first apply average pooling smooth M and get (M). Then we select the point with the highest relevancy in Aug(M .M) as the predicted point. Additionally, we choose the semantic level corresponding the highest relevancy value among the three levels the selected level. For semantic segmentation, after getting Aug( MM) we select the level in the same way as object localization. We then normalize Aug(M) and use threshold to obtain binary mask the final prediction mask.
3D-OVS. The post-processing pipeline for semantic segmentation is similar to that used for the LERF dataset. Differently, we obtain the smoothed relevancy map Avg'(M) as Aug' (M) 0.5 x (M + Avg (M)), and the level selection is based on the average relevancy within the predicted mask. Additionally, when evaluating query accuracy, we use two coarse- -ranined semantic levels (whole and parl) owing to the simplicity of the scenes in this dataset.
Mip-NeRF360. Similar to the post-processing for the 3D-OVS dataset, we obtain the smoothed relevancy map Aug((M) and select the semantic level based on the average relevancy score. When
23

---

13 and evaluating query accuracy, we use all three semantic levels, as the complex scenes in this dataset demand finer-grained language feature rendering.
D More Ablation Study
Time Analysis. We conduct ablation studies on the LERF 13] dataset and report the render and decode speed (ms per query) in Table 9 We test the speed on one A100 GPU. The baseline LangSplat, which renders three-level ttrre-leve_--dimmmiiiona language features separately. Rendering three semantic levels in parallel can reduce the rendering time from 6.0 ms to 2.0 ms per query. The sparse coefficient field can significantly speed up the decoding stage from 83.1 ms/q to 0.1 ms/q by changing the heavy weight decoder to a simple matrix multiplication operation. Efficient sparse coefficient splatting method effectively transformed the 192-dimensional rendering into 12- dimensional rendering, reducing the rendering time from 5.3 ms to 2.0 ms per query through CUDA optimization.
Table 9: Speed ablation study on the LERF dataset. Parallel means rendering three semantic levels in parallel, Sparse means 3D sparse coefficient field, Efficient means efficient sparse coefficient splatting with CUDA optimization.
Component
Speed (ms/q)
Parallel
Sparse
Efficient
render
decode
total
6.0
83.1
89.1
2.0
83.1
85.1
0.1
5.4
2.0
0.1
2.1
and K. Table 10 and Table 11 show the ablation study results on each scene in the LERF [13] dataset. As we can see, for some complex scenarios, such as Kitchen and Figurines, larger and K can lead better performance. However, for all scenarios, the model's 's performance is already good enough with the settings of L = 64 and K = 4, and increasing L and K will lead to increased computational resources and time cost for training and inference. Therefore, we L 64
and K = 4 to balance performance and efficiency.
Table 10: Ablation study on codebbok size L. We report the localization and the segmentation performance on the LERF dataset.
L
32
64
128
Accuracy (%)
70.4
74.7
74.7
Ramen
loU (%)
50.5
51.8
51.4
Accuracy (%) loU (%)
84.8
93.2
91.5
Teatime
65.7
72.2
69.8
Accuracy (%) IoU (%)
68.2
86.4
86.4
Kitchen
54.2
59.1
63.2
Accuracy (%) IoU (%)
67.9
82.1
83.9
Figurines
45.3
56.4
57.6
Accuracy (%) loU (%)
72.8
84.1
84.1
Overall
53.9
59.9
60.5
E More Visualization Results
Open-vocabulary 3D Object Localization. We visualize more examples on the LERF
Mip-NeRF360 [13] datasets for open-vocabulary 3D object localization in Figure

---

Table 11: Ablation study on different K. report the localization and the segmentation performance on the LERF dataset.
6
8
Accuracy (%)
71.8
74.7
71.8
73.2
Ramen
IoU (%)
50.2
51.8
51.4
51.1
Accuracy (%)
91.5
93.2
91.5
91.5
Teatime
IoU (%)
68.8
72.2
70.7
70.6
Accuracy (%)
77.3
86.4
95.5
90.9
Kitchen
IoU (%)
48.9
59.1
59.8
60.1
Accuracy (%)
76.8
82.1
78.6
78.6
Figurines
IoU (%)
49.6
56.4
57.7
57.6
Accuracy (%)
79.4
84.1
84.4
83.6
Overall
IoU (%)
54.4
59.9
59.9
59.9
Open-vocabulary 3D Semantic Segmentation. We visualize more examples on the LERF [13]
Mip-NeRF360 (66] and 3D-OVS 47 datasets for open-vocabulary 3D semantic segmentation in Figure
25


Reviews:
[{"review_id": "CmqvtBmAwS", "reviewer": "Reviewer_T9NY", "summary": "The paper presented LangSplatV2, an approach for high-dimensional language feature splatting in 3D open-vocabulary querying, aiming to address the speed limitations of LangSplat. By identifying the decoding stage in LangSplat to be the bottleneck, a 3D sparse coefficient field field is proposed to represent the language embedding of each Gaussian point, coupled with an efficient scheme to learn such coefficients, LangSplatV2 obtained significant speed up compared to LangSplat.", "questions": "What are input image resolutions of datasets that evaluation was conducted? How sensitive is LangSplatV2 regarding this?", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "[Strengths]\n\n1) Thorough analysis performed to identify computation bottleneck in LangSplat.\n2) Representing each Gaussian point with a sparse coefficient directly learned from codebook in 3D space is novel and effective.\n3) Extensive experiments demonstrated that LangSplatV2 achieved solid improvements across a number of tasks compared to previous methods. \n\n[Weakness]\n\nI do not see major weakness of the proposed LangSplatV2. Although if runtime analysis when compared with other methods can be included, it would give a more wholistic picture. It would also be good to see code open-sourced.", "was_revised": true, "final_justification": "Given the authors' rebuttal has addressed my questions, I maintain my rating \"Borderline Accept\".", "rebuttal": "Thank you for the constructive feedback and the positive assessment of our work! Below, we detail our responses to the review questions.\n\n**Q1: Runtime Analysis**\n\nThanks for the suggestion. We provide a detailed runtime comparison in the table below.\n\n\n| Method | Render(ms) | Decode(ms) | Post(ms) | Total(ms) | FPS |\n|----------|----------|----------|-------|-----|-----|\n| GS-Grouping  | 4.4      | 0.5       | 659.5 | 664.4 | 1.5 |\n| LEGaussian   | 9.8      | 4.3       | 22.6 | 36.7 | 27.2 |\n| LangSplat*    | 2.0      | 83.1      | 0.5 | 85.6 | 11.7 |\n| **LangSplatV2**|**2.0**|**0.1**|**0.5**|**2.6**|**384.6**|\n\nThe results are obtained on the LERF dataset with one A100 GPU. It can be observed that LangSplatV2 achieves the fastest speed at every stage, leading to a substantial improvement in inference efficiency. Furthermore, Tables 2, 3, and 4 in the main paper demonstrate that LangSplatV2 attains state-of-the-art performance across multiple datasets. In summary, LangSplatV2 not only achieves superior or competitive query accuracy but also exhibits significantly higher speed.\n\n\n**Q2:It would also be good to see code open-sourced.**\n\nYes, we will release the complete source code, training scripts, and model weights to ensure full reproducibility upon acceptance.\n\n\n\n\n**Q3:Input Image Resolutions**\n\n\n\nThank you for the question. The input image resolutions used in our evaluation are: 730×980 for LERF, 1080×1440 for 3D-OVS, and 1080×1620 for Mip-NeRF360. These datasets cover a wide range of resolution settings. As shown in the table below, our method LangSplatV2 consistently outperforms prior approaches such as GS-Grouping and LEGaussian in terms of segmentation accuracy, query speed, and memory efficiency across all these datasets. Despite the varying input resolutions, LangSplatV2 maintains real-time query performance (as low as 2.6 ms on LERF, 4.8 ms on 3D-OVS, and 3.8 ms on Mip-NeRF360) while also achieving the highest Segmentation IoU on all three benchmarks. This demonstrates that our method is robust to input resolution changes and can deliver high-quality, efficient performance across different scene configurations.\n\n\n| LERF(730 $\\times$ 980) | Seg(IoU) |Query Time(ms)|Memory(GB)|\n|-------------|---------------------|------|-----|\n| GS-Grouping | 36.6 | 664.4 | 7.3  |\n| LEGaussian  | 24.6 | 36.7 | 8.2 |\n| **LangSplatV2** | **59.9** | **2.6** | **7.2** |\n\n| 3D-OVS(1080 $\\times$ 1440) | Seg(IoU) |Query Time(ms)|Memory(GB)|\n|-------------|---------------------|------|-----|\n| GS-Grouping | 87.7 | 1016.4 | 7.6  |\n| LEGaussian  | 88.5 | 59.6 | 10.5 |\n| **LangSplatV2** | **94.6** | **4.8** | **11.6** |\n\n\n| Mip-NeRF360(1080 $\\times$ 1620) | Seg(IoU) |Query Time(ms)|Memory(GB)|\n|-------------|---------------------|------|------|\n| GS-Grouping | 49.2 | 1120.2 | 12.0  |\n| LEGaussian  | 29.1 | 58.0 | 14.0 |\n| **LangSplatV2** | **69.4** | **3.8** | **16.9** |"}, {"review_id": "E6yncIjLUT", "reviewer": "Reviewer_Vun8", "summary": "Summary: \n- In this work authors build LangSplatv2, which results in significant (more than 450 times!) speedup of the prior work LangSplat. \n- The primary contribution is improving the speed bottleneck of the prior work, which is decoding the low dimension textual embedding to high dimension (in CLIP space). \n- The speed up comes from using a sparse (K-dim) 3D coefficient field and a global codebook (of total L basis vectors) to represent the language field. \n- To convert the sparse coefficients to dense embedding, authors use top-K pruning and softmax normalization.\n- Authors conduct exhaustive experiments comparing performance against baselines and several scenes, as well as benchmark their timings.", "questions": "- Will you open-source the code? Since majority gains in this work come from low-level optimizations (parallelizing alpha-blending, etc) I believe this would be very helpful.", "limitations": "Yes.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- I believe this work takes the next right step in optimizing for the speed-up of a pipeline like LangSplat, which can enable several open-ended downstream applications. \n- The gains are impressive - both in terms of speed-up, 3D localization and semantic segmentation performance as well. \n- Experiments are thorough and compare against many existing baselines, ablation are comprehensive including timing studies. \n- Paper is written well, and the figures are neat and helpful. \n\nWeaknesses:\n- The only major issue is - I was hoping to find videos in the supplementary, but couldn’t find any - it would’ve been helpful to see the visuals and qualitatively compare against said baselines. \n- Could you demonstrate visually where the LangSplat vs LangSplatv2 performance gap in the Kitchen scene appears from? I want to understand the claim on L298 better, from Figure 5. LangSplatV2 appears to be better. \n- The training time seems to have tripled compared to LangSplat, I wonder if the authors tried any workarounds for this?", "was_revised": true, "final_justification": "I thank the authors for providing detailed explanation, and clarifications. \n\nI will keep my original score of Accept for this work.", "rebuttal": "Thank you for the constructive feedback and the positive assessment of our work! Below, we detail our responses to the review questions.\n\n\n**Q1: Video Demo**\n\nThank you for the suggestion. Due to NeurIPS rebuttal guidelines, we are unable to upload images or videos during the rebuttal phase. However, we will publicly release video demos that show visual comparisons with the baselines upon acceptance.\n\n**Q2: Performance Gap in the Kitchen Scene**\n\nThanks for the question. Actually LangSplatV2 attains a more accurate 3D language field in the Kitchen scene. \n\nTo better understand the performance gap, we encourage referring to Table 2 of our main paper, where LangSplatV2 significantly outperforms LangSplat in the 3D Semantic Segmentation task (59.1% vs. 44.5%). This task provides a more reliable and comprehensive assessment of the learned 3D language field, as it considers the overall semantic consistency of the entire 3D space rather than relying on a single point prediction.\n\nIt's important to note that the 3D Object Localization task uses the location with the maximum similarity as the predicted object center, which can be disproportionately affected by noise and local artifacts. In the Kitchen scene specifically, this sensitivity—combined with the limited number of test samples—results in high variance. In fact, when we ran multiple evaluations on the Kitchen scene, the average localization accuracy of LangSplat was 79.5%, compared to 86.4% for LangSplatV2, demonstrating the superiority of LangSplatV2.\n\nFinally, Figure 5 qualitatively validates that LangSplatV2 exhibits more semantically accurate and spatially coherent activations, reinforcing that it indeed learns a better 3D language field in the Kitchen scene.\n\n\n**Q3: Workarounds for the Training Time**\n\nThank you for your question. The increased training time compared to LangSplat primarily stems from the need to learn high-dimensional 3D semantic fields in LangSplatV2, which naturally incurs more computational overhead. We acknowledge this trade-off and have actively explored ways to mitigate it.\n\nAmong the techniques we tested, one of the most effective was the initialization strategy for the global codebook. Instead of using random initialization, we first extract all semantic features from the input 2D images and perform clustering. The resulting cluster centers are then used to initialize the codebook. This initialization significantly improves convergence speed.\n\nAs shown in the table below, training with this initialization (denoted as “w init”) achieves much lower training loss at early iterations compared to the version without initialization (“w/o init”). Specifically, with only 10K iterations, the “w init” version reaches a similar loss level as the “w/o init” version after 30K iterations. As a result, we were able to reduce LangSplatV2’s training iterations from 30K (used in LangSplat) to just 10K, effectively narrowing the training time gap. We recognize that training efficiency remains an important concern and plan to further investigate strategies to accelerate training. We believe this will be a valuable and interesting direction for future work.\n\n|    Training Loss     | 1k   | 2K   | 4K   | 6k   | 8k   | 10k  | 15k  | 20k  | 25k  | 30k  |\n|---------|------|------|------|------|------|------|------|------|------|------|\n| w init  | 0.191| 0.181| 0.177| 0.172| 0.172| 0.168|  -   |  -   |  -   |  -   |\n| w/o init| 0.260| 0.210| 0.195| 0.189| 0.181| 0.175| 0.170| 0.169| 0.169| 0.168|\n\n**Q4:Will you open-source the code?**\n\nWe commit to releasing the complete source code, training scripts, and model weights to ensure full reproducibility upon acceptance.\n\n**Q5:Typo**\n\nThanks! We will fix it in the revised version."}, {"review_id": "CUrzbnJLly", "reviewer": "Reviewer_yuhH", "summary": "Paper Motivation:\n\nThe paper addresses the task of interacting with 3d scenes with natural language queries. \nFor this the scene must be augmented with language features (e.g. CLIP features). Previous work LangSplat allows to embed CLIP features to scenes reconstructed with 3D Gaussian Splatting.\nBut LangSplat does not achieve real-time performance, limiting its applications.\n\nContribution:\n\nThe authors propose to use codebook learning to improve rendering speed of 3DGS with language features:\n- during reconstruction a global codebook is learned \n- each Gaussian has learnable coefficients for the entries in the codebook\n- only top k coefficients are used to increase performance\n- during rendering the coefficients are rasterized into a feature image \n- per pixel CLIP features are computed as linear combination of feature image and codebook\n\nResults:\n\nAuthors show that their method archives real time rendering with SOTA accuracy on multiple datasets.\nResults are compared to existing methods and an ablation study is performed to measure the effect of introduced hyper-parameters.", "questions": "The paper is well written and in a very good state. My main concern is the related work section which seams like an arbitrary list of work related to 3DGS in some parts and misses related work which also aims to speed up rendering of 3DGS.\n\nIf the authors address my concerns about the related work I will increase my score.", "limitations": "Limitations are adequately addressed.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n- paper is well written and has an easy to follow clear structure\n- graphics are well made and help to better understand the method \n- addresses major limitation (performance) of existing methods without reducing accuracy (!!!)\n- novel approach for accelerating rendering of high dimensional features\n- authors promise to release code\n\nWeaknesses:\n- related work seems minimal / short and can be improved\n- Paper focuses on making 3DGS rendering faster, related work should include papers with similar goals.\n- specifically, other works  (e.g. Compressed 3D Gaussian Splatting) have used codebooks to accelerate 3DGS rendering. The authors discuss this in the appendix but this should definitely be moved to related work in the main paper.  \n- training is 3x slower compared to LangSplat. IMO this should be mentioned with numbers in the paper (not just reference to appendix)", "was_revised": true, "final_justification": "The authors have adequately addressed my concerns regarding the lack of related work and I therefore raise my score to accept (5).", "rebuttal": "Thank you for the constructive feedback and the positive assessment of our work! Below, we detail our responses to the review questions.\n\n**Q1: My main concern is the related work section which seams like an arbitrary list of work related to 3DGS in some parts and misses related work which also aims to speed up rendering of 3DGS.**\n\nThank you for the helpful suggestion. We will revise and expand the Related Work section in the main paper accordingly. Specifically, we will reorganize the current content and include a dedicated paragraph discussing 3D Gaussian Splatting acceleration, including relevant works such as Compressed 3D Gaussian Splatting that also adopt codebook-based strategies for faster rendering. We will include the following discussion in the related work section.\n\n\n>To reduce computational overhead and improve rendering speed, recent works have proposed various strategies to accelerate 3D Gaussian Splatting (3DGS). One line of work compresses the Gaussian representation using vector quantization and codebooks. For instance, Compressed 3D Gaussian Splatting [1] adopts sensitivity-aware clustering and quantization-aware training to compress both directional colors and Gaussian parameters, achieving up to 31× compression and 4× rendering speedup. CompGS [2] similarly applies K-means-based quantization to achieve 40–50× storage reduction and 2–3× faster rendering with minimal quality loss.\n\n>Other approaches focus on structural regularization or architectural simplification. Self-Organizing Gaussian Grids [3] reorganize Gaussian parameters into spatially coherent 2D grids, reducing redundancy via local homogeneity. LightGaussian [4] uses pruning based on global significance and vector quantization, achieving 15× compression and 200+ FPS rendering. [5] proposes to reduce the number of Gaussians via a learnable mask and introduces compact view-dependent color encoding, achieving a 25× reduction in storage. Techniques like Speedy-Splat [6] and SORT-Free Splatting [7] improve runtime performance through optimized rasterization and differentiable approximations of alpha blending.\n\n>In contrast to these works that focus on accelerating RGB-based scene rendering, our method tackles the unique challenge of handling high-dimensional Gaussian features (e.g., 512D). Extending prior compression techniques [1, 4] to our setting would require learning and quantizing such high-dimensional attributes per Gaussian, leading to excessive memory usage. Moreover, directly rendering with full 512D features remains computationally demanding. In contrast, our proposed method circumvents the need to directly learn and render high-dimensional features for 3D Gaussian points.\n\n[1] Niedermayr, Simon, Josef Stumpfegger, and Rüdiger Westermann. \"Compressed 3d gaussian splatting for accelerated novel view synthesis.\" CVPR 2024.\n\n[2] Navaneet, K. L., et al. \"Compgs: Smaller and faster gaussian splatting with vector quantization.\" ECCV 2024.\n\n[3] Morgenstern, Wieland, et al. \"Compact 3d scene representation via self-organizing gaussian grids.\" ECCV 2024.\n\n[4] Fan, Zhiwen, et al. \"Lightgaussian: Unbounded 3d gaussian compression with 15x reduction and 200+ fps.\" NeurIPS 2024.\n\n[5] Lee, Joo Chan, et al. \"Compact 3d gaussian representation for radiance field.\" Proceedings of the CVPR 2024.\n\n[6] Hanson, Alex, et al. \"Speedy-splat: Fast 3d gaussian splatting with sparse pixels and sparse primitives.\" CVPR 2025.\n\n[7] Hou, Qiqi, et al. \"Sort-free gaussian splatting via weighted sum rendering.\" ICLR 2025.\n\n\n\n**Q2: Training numbers should be mentioned in the paper (not just reference to appendix)**\n\nThanks for the suggestion. We will mention it and provide a detailed analysis in the main paper in the revised version."}, {"review_id": "lvu1heVcBF", "reviewer": "Reviewer_w8ui", "summary": "This paper focuses on addressing the speed bottleneck of existing 3D language field models, particularly exemplified by LangSplat, when performing open-vocabulary queries. Through detailed performance analysis, the authors identify that the primary cause of the slowdown lies in the model’s heavy multi-layer perceptron (MLP) decoder. This component is responsible for reconstructing high-dimensional CLIP features from low-dimensional latent representations during inference and consumes the vast majority of computational time, thereby limiting the potential for real-time applications.\n\nTo overcome this limitation, the authors propose a new model called LangSplatV2, which introduces two key technical innovations. First, they present a novel representation method termed the 3D Sparse Coefficient Field. The core idea is based on the assumption that the semantic content of a scene is inherently limited, allowing the high-dimensional language features of millions of Gaussian points to be expressed as sparse linear combinations of basis vectors drawn from a globally shared codebook. Instead of learning individual low-dimensional features for each Gaussian point, the model learns sparse coefficients that combine these basis vectors efficiently. This approach completely eliminates the need for the heavy MLP decoder during inference, replacing it with a much faster matrix multiplication operation.\n\nSecond, to maintain efficiency during rendering despite the introduction of the codebook, the authors design and implement an efficient splatting technique using CUDA optimization. This method exploits the sparsity of the learned coefficients and only processes the K non-zero values during rendering—where K is significantly smaller than the total codebook size L. As a result, the computational cost of rendering a high-dimensional feature map, such as one with 1536 dimensions, becomes comparable to that of rendering an ultra-low-dimensional map of just 12 dimensions. This effectively decouples the rendering speed from the final feature dimensionality.\n\nWith these improvements, LangSplatV2 achieves a dramatic speedup over the original LangSplat model—up to tens of times faster—while maintaining or even improving query accuracy. For instance, the 3D open-vocabulary query speed increases from 8.2 FPS to 384.6 FPS. This breakthrough enables real-time interaction with high-resolution, complex 3D scenes. Extensive experiments conducted on several public datasets, including LERF, 3D-OVS, and Mip-NeRF360, confirm the model’s substantial advantages in both speed and accuracy.", "questions": "**1. Quantitative Analysis of Training Cost**\n\nThe paper acknowledges in the conclusion that the performance gains of LangSplatV2 come “at the cost of increased training cost.” This is a crucial trade-off, yet no quantitative data is provided in the main text. To allow reviewers and readers to fully understand the overall cost of your method, please provide a quantitative comparison of training overhead between LangSplat and LangSplatV2. It is recommended that you report and compare the following metrics on one or more representative datasets (e.g., LERF):\n    a. Total training time required to reach convergence.  \n    b. Peak GPU memory usage during training.  \n    c. Number of training iterations required to achieve comparable performance.\n\n**2. Statistical Significance of Model Accuracy Improvements**\n\nThe paper reports accuracy improvements over the baseline in several tables. However, some of these improvements are relatively small (for example, on the 3D-OVS dataset, the overall mIoU increases from 93.4% to 94.6% ). Moreover, no evidence of statistical significance is provided. You yourselves note that the baseline model exhibits high variance, which further highlights the necessity of statistical validation. Could you please validate the stability of key metrics through multiple runs (e.g., using 3–5 different random seeds), and report the mean and standard deviation?\n\n**3. Further Clarification of Originality and Comparison with Concurrent Work**\n\nOne of the core ideas in the paper is the use of a codebook, and a concurrent work, `LEGaussian` , adopts a similar approach. Although you attempt to differentiate your method in the discussion section, a deeper elaboration is needed. Could you please more clearly explain the fundamental differences and advantages of LangSplatV2 compared to `LEGaussian`? Specifically, you mention that `LEGaussian` “still relies on an MLP to compress features and does not exploit sparsity.” Please elaborate on how your proposed “Efficient Sparse Coefficient Splatting” method offers technical and practical advantages over other codebook-based approaches in both implementation and final performance?", "limitations": "Yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "### Strengths of the Paper\n\n1. **Addresses a clear and important problem:** The paper successfully tackles the real-time performance bottleneck of the 3D language model `LangSplat`. Increasing the query speed from 8.2 FPS to 384.6 FPS represents a significant engineering achievement, with substantial practical value for applications requiring real-time feedback, such as robotics and augmented reality.\n\n2. **The method is cleverly designed and well-targeted:** The paper starts from a clear motivation—identifying the decoder as the main performance bottleneck through rigorous performance analysis. The proposed solution, which replaces low-dimensional features and the decoder with a sparse coefficient field and global codebook, directly and effectively addresses this core issue.\n\n3. **Extensive experimental validation:** The authors evaluate their model on multiple public datasets including LERF, 3D-OVS, and Mip-NeRF360, demonstrating the generalizability of their approach . The results also show that while significantly improving speed, the model maintains or even slightly improves segmentation and localization accuracy across most scenarios .\n\n---\n\n### Weaknesses of the Paper\n\n1. **Limited originality:** While the specific implementation and optimizations are novel, the core concept—using a codebook to represent Gaussian features—is not entirely new. The paper itself references a concurrent work, `LEGaussian`, which explores similar ideas.  This somewhat diminishes its theoretical novelty.\n\n2. **Notable shortcomings in experimental evaluation:**\n   - **Lack of training cost analysis:** The paper acknowledges in the conclusion that the method “increases training cost” , which is a crucial trade-off. However, no quantitative data (e.g., training time, peak GPU memory usage) comparing it to `LangSplat` is provided in the main text. This omission prevents readers from fully evaluating the overall cost-benefit trade-off and may give the impression that the paper selectively highlights inference advantages. A thorough analysis of the costs associated with the proposed method is essential for a rigorous scientific study.\n   - **Absence of statistical significance testing:** All reported accuracy and IoU metrics in the paper are based on single runs without error bars or multiple trials to establish statistical confidence. The authors themselves note that `LangSplat` exhibits high variance in certain scenarios, underscoring the importance of statistical validation. Without it, the credibility of claims regarding “accuracy improvement” is significantly weakened. Readers cannot determine whether small performance gains (e.g., an increase from 93.4% to 94.6% on 3D-OVS) reflect consistent improvements from the method or random fluctuations.\n\n3. **Depth of contribution:** Although the performance gains are impressive, the paper can be viewed more as excellent \"engineering optimization\" rather than a \"theoretical breakthrough,\" suggesting a lack of conceptual depth.\n\n---\n\n### Summary and Recommendations\n\n**Overall Assessment:** This paper proposes a highly effective optimization technique that dramatically improves the inference speed of 3D language Gaussian splatting models, making them viable for real-time applications—an achievement that deserves recognition.\n\nHowever, the paper falls short in terms of evaluation rigor and originality. The omission of training cost analysis—a key trade-off—and the lack of statistical support for accuracy claims constitute major scientific shortcomings. Furthermore, the overlap in core ideas with contemporary work reduces the paper’s claim to originality.", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the constructive feedback and the positive assessment of our work! Below, we detail our responses to the review questions.\n\n\n**Q1:Quantitative Analysis of Training Cost**\n\nBelow, we provide a quantitative comparison of training cost between LangSplatV2 and other methods on the LERF dataset, using a single NVIDIA A100 GPU. We report the total training time until convergence, peak GPU memory usage, and the number of training iterations needed to reach comparable performance:\n\n| Method | Training Time(h)|Training Iteration|Peak GPU Memory(GB)|\n|-------------|---------------------|------|------|\n| LEGaussian | 1.3 | 30K  | 11 |\n| LangSplat  | 1.0 | 30K | 6.2 |\n| LangSplat(512 Dim) | 45.0 | 30K | 47.4 | \n| **LangSplatV2** | **3.0** | **10K** | **21.2**|\n\nAs we can see, LangSplatV2 indeed incurs a higher training cost compared to LangSplat and LEGaussians due to the need to construct and optimize high-dimensional semantic fields. However, our main contribution lies in significantly improving test-time performance, which is more critical for real-world deployment.\n\nSpecifically, LangSplatV2 achieves 47× faster inference than LangSplat and 14× faster than LEGaussians under similar memory usage. It also yields higher 3D semantic segmentation accuracy, demonstrating a better quality-performance trade-off.\n\nAdditionally, compared to the training cost of a naive version of LangSplat trained directly with 512D language features, LangSplatV2 is more efficient (3h vs. 45h), thanks to the introduction of a global codebook and sparse coefficient field, which reduces training complexity without sacrificing feature expressiveness.\n\n\n\n**Q2:Statistical Significance of Model Accuracy Improvements**\n\nThank you for the suggestion. Following your advice, we conducted additional experiments on the LERF and 3D-OVS datasets using different random seeds to evaluate the stability and statistical significance of our results. For each scene, we repeated the training of LangSplatV2 three times and report the mean and standard deviation of the mIoU scores in the table below.\n\n| LERF Scene     | Exp.1 | Exp.2 | Exp.3 | mean(±std) | LangSplat |\n|-------------|---------------------|------|-----|----|----|\n| Teatime   | 72.2 | 71.2 | 71.5 | 71.6(±0.5) | 65.1 |\n| Ramen     | 51.8 | 51.8 | 51.5 | 51.7(±0.1) | 51.2 |\n| Figurines | 56.4 | 56.9 | 57.2 | 56.8(±0.4) | 44.7 |\n| Kitchen   | 59.1 | 58.4 | 62.9 | 60.1(±2.4) | 44.5 |\n| Overall   | 59.9 | 59.6 | 60.8 | 60.1(±0.6) | 51.4 |\n\n| 3D-OVS Scene     | Exp.1 | Exp.2 | Exp.3 | mean(±std) | LangSplat |\n|-------------|---------------------|------|-----|----|----|\n| Bed     | 93.0 | 93.0 | 94.4 | 93.5(±0.8) | 92.5 |\n| Bench   | 94.9 | 95.0 | 94.7 | 94.9(±0.2) | 94.2 |\n| Room    | 96.1 | 95.7 | 95.7 | 95.8(±0.2) | 94.1 |\n| Sofa    | 92.3 | 91.9 | 92.2 | 92.1(±0.2) | 90.0 |\n| Lawn    | 96.6 | 96.7 | 96.6 | 96.6(±0.1) | 96.1 |\n| Overall | 94.6 | 94.5 | 94.7 | 94.6(±0.1) | 93.4 |\n\n\nAs shown, LangSplatV2 exhibits low variance in most scenarios. In cases where relatively higher variance is observed, such as in the Kitchen and Bed scenes, all repeated runs still significantly outperform the LangSplat baseline. Overall, these results validate the statistical significance and consistency of our reported improvements, and reinforce the robustness of LangSplatV2 under different random initializations.\n\n**Q3: Further Clarification of Originality and Comparison with Concurrent Work**\n\nThank you for the comment. We respectfully disagree with the implication that our method is similar to LEGaussian. While both methods utilize a codebook-based representation, LangSplatV2 is fundamentally different in design and motivation. It is precisely this novel direction that enables us to achieve significantly better rendering speed and accuracy compared to prior work. Below, we elaborate on the key differences and technical advantages of LangSplatV2 over LEGaussian:\n\n**1. Global 3D Codebook vs. 2D Codebook**\n\nLEGaussian first builds a 2D codebook by quantizing features in the image space, then learns a 3D model to predict the index of the codebook via classification. In contrast, LangSplatV2 directly learns a global 3D codebook shared across all Gaussians in true 3D space. This allows our method to learn quantization directly in the 3D spatial domain, rather than in projected 2D images, resulting in lower quantization error and more accurate 3D representation modeling.\n\n**2. No MLP Bottleneck**\n\nLEGaussian relies on an MLP to map Gaussian features to a codebook index, which introduces computational overhead and acts as a bottleneck. LangSplatV2 eliminates the MLP altogether, removing this bottleneck and achieving much faster rendering and training performance.\n\n**3. Efficient Sparse Coefficient Splatting**\n\nWe propose a new Efficient Sparse Coefficient Splatting mechanism, which assigns only a small number of nonzero coefficients per Gaussian, reducing the effective rendering dimension to $K$. In contrast, LEGaussian renders with full high-dimensional feature vectors, which is less efficient and limits scalability in practice.\n\nMoreover, LEGaussian operates in a compressed latent space and does not preserve the full expressiveness of high-dimensional language features. Its 2D codebook design also fails to fully leverage 3D spatial priors. LangSplatV2, by contrast, learns and renders high-dimensional language features directly in 3D, leading to more semantically meaningful and accurate reconstructions.\n\n\nAs shown in Table 7 of our main paper, LangSplatV2 achieves superior semantic segmentation accuracy, faster inference speed, and comparable memory usage across three benchmark datasets, compared with LEGaussian, demonstrating the practical impact of our novel formulation."}]

Author rebuttal:

Meta-review: This paper achieves 47x speedup over LangSplat (476 FPS) by replacing the MLP decoder with a sparse coefficient field and global codebook. Four reviewers with three engaged post-rebuttal. One reviewer raised the score to 5, while another remained unresponsive. The proposed method demonstrates significant practical impact, impressive speedup that maintains accuracy, and thorough experiments on LERF, 3D-OVS, and Mip-NeRF360 datasets. Authors addressed concerns, including theoretical novelty (clarified key differences from LEGaussian), 3x training cost (optimization reduces iterations from 30k to 10k), and statistical validation (additional experiments show 94.6 mIoU) during the rebuttal. Despite limited theoretical novelty, the substantial engineering contribution enables real-time applications. Therefore, the AC recommends Acceptance as a poster paper.
Decision: Accept (poster)