FHGS: Feature-Homogenized Gaussian Splatting
Authors: Q.G. Duan
Benyun ZHAO
Mingqiao Han
Yijun Huang
Ben M. Chen
Venue: NeurIPS 2025 poster ()
Keywords: 3D Gaussian Splatting;Feature Fusion;Scene Understanding;
Primary area: general_machine_learning
TL;DR: we proposes a novel 3D feature distillation framework inspired by physical models, which freezes and distills 2D pre-trained features into 3D representations while preserving the real-time rendering efficiency of 3DGS.

Abstract: Scene understanding based on 3D Gaussian Splatting (3DGS) has recently achieved notable advances. Although 3DGS related methods have efficient rendering capabilities, they fail to address the inherent contradiction between the anisotropic color representation of gaussian primitives and the isotropic requirements of semantic features, leading to insufficient cross-view feature consistency.
To overcome the limitation, we proposes FHGS (Feature-Homogenized Gaussian Splatting), a novel 3D feature distillation framework inspired by physical models, which freezes and distills 2D pre-trained features into 3D representations while preserving the real-time rendering efficiency of 3DGS.
Specifically, our FHGS introduces the following innovations: Firstly, a universal feature fusion architecture is proposed, enabling robust embedding of large-scale pre-trained models' semantic features (e.g., SAM, CLIP) into sparse 3D structures.
Secondly, a non-differentiable feature fusion mechanism is introduced, which enables semantic features to exhibit viewpoint independent isotropic distributions. This fundamentally balances the anisotropic rendering of gaussian primitives and the isotropic expression of features; Thirdly, a dual-driven optimization strategy inspired by electric potential fields is proposed, which combines external supervision from semantic feature fields with internal primitive clustering guidance. This mechanism enables synergistic optimization of global semantic alignment and local structural consistency.
Extensive comparison experiments with other state-of-the-art methods on benchmark datasets demonstrate that our FHGS exhibits superior reconstruction performance in feature fusion, noise suppression, and geometric precision, while maintaining a significantly lower training time.
This work establishes a novel Gaussian Splatting data structure, offering practical advancements for real-time semantic mapping, 3D stylization, and Vision-Language Navigation (VLN).
Our code and additional results are available on our project page:https://fhgs.cuastro.org/.

FHGS: Feature-Homogenized Gaussian Splatting
Q. G. Duan Benyun Zhao Mingqiao Han
Yijun Huang Ben M. Chen
Department of Mechanical and Automation Engineering The Chinese University of Hong Kong
Abstract
Scene understanding based on 3D Gaussian Splatting (3DGS) has recently achieved notable advances. Although 3DGS related methods have efficient rendering capa- bilities, they fail to address the inherent contradiction between the anisotropic color representation of gaussian primitives and the isotropic requirements of semantic features, leading to insufficient cross-view feature consistency. To overcome the limitation, we proposes FHGS
Gaussian Splatring). novel 3D feature distillation framework inspired by physical models, which freezes and distills 2D pre-trained features into 3D representations while preserving the real- time rendering efficiency of 3DGS. Specifically, our FHGS introduces the following innovations: Firstly, universal feature fusion architecture is proposed, enabling robust embedding of large-scale pre-trained models' semantic features (e.g.. SAM, CLIP) into sparse 3D structures. Secondly, a non-differentiable feature fusion mechanism is introduced, which enables semantic features to exhibit viewpoint independent isotropic distributions. This fundamentally balances the anisotropic rendering of gaussian primitives and the isotropic expression of features; Thirdly, a dual-driven optimization strategy inspired by electric potential fields proposed, which combines external supervision from semantic feature fields with internal primitive clustering guidance. This mechanism enables synergistic optimization global semantic alignment and local structural consistency. Extensive comparison experiments with other state-of the-art art methods on benchmark datasets demon- strate that our FHGS exhibits superior reconstruction performance in feature fusion, noise suppression, and geometric precision, while maintaining a significantly lower training time. This work establishes novel Gaussian Splatting data structure, of- fering practical advancements for real-time semantic mapping, 3D stylization, and Lision-Language Navigation (VLN). Our code and additional results available on our project page: pottts://fhgeee cuastro. org/
Introduction
In recent years, scene representation particularly understanding has emerged prominent research focus, as it enables unmanned systems to better perceive and interpret their surrounding environments. Traditional scene representation frameworks such Multi-View Stereo (MVS) [I] and Simultancous Localization and Mapping (SLAM) can achieve geometric reconstruction. However, these methods rely non-differentiable pipelines and remain limited in high-level semantic perception and nonlinear feature fusion. As a result, the differentiable approaches have gradually come into focus. Among them, Neural Radiance Fields (NeRF) 3 and 3D Gaussian Splatting (3DGS) [4] have revolutionized the scene representation framework. NeRF models implicit radiance fields and learns continuous 3D spatial representations under 2D image supervision via differentiable volume rendering
CCorresponiigg author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

(73)
ema
Fg (n)
SHI g
c
BORSE
Feature Field: F gt (n)
Pm (n) (n) =
RGB Field: mm c
Obverse
Reverse
Figure I: The left part demonstrates the inherent contradiction between the anisotropic color c of gaussian primitives in RGB field and the isotropic requirement of semantic features f. The right part shows the results in obverse and reverse, indicating that FHGS shows superior reconstruction performance in terms of feature fusion, noise suppression, and geometric accuracy.
equations, whereas 3DGS adopts explicit anisotropic Gaussian primitives enable high-quality reconstruction through efficient rasterization. However, these traditional neural field representations primarily focus on the fusion of RGB geometric fields, with limited exploitation of semantic features. In contrast, feature fields require maintaining semantic consistency across multiple viewpoints to prevent contradictory predictions during viewpoint transitions
(5, as shown in Fig. |
Against this backdrop, the integration of semantic feature from transformer-based models 6 7 fusing with NeRF and 3DGS framework has begun to emerge. NeRF-based frameworks extend radiance fields by incorporating learnable semantic feature fields. implicitly enforcing multi-view semantic consistency through continuous neural representations [8] [9. However, their inference speed remains limited due to the dense sampling required by volumetric rendering. In contrast, 3DGS- based frameworks [0] construct explicit feature fields by directly associating semantic features with their corresponding explicit primitives. However, as shown in the Fig. [ an inherent conflict arises between the anisotropic nature of RGB fields their rasterization pipeline and the isotropic representation required for robust semantic features. Existing methods, whether based on implicit or explicit representations, typically treat features as fully differentiable and optimize them jointly with appearance. However, this continuous optimization may introduce inconsistencies that interfere with the self-attention mechanism in transformer, lcading to feature noise and degraded rendering quality.
To address the limitations of aforementioned phenomena, we propose FHGS Gaussian Splatting). a novel feature fusion framework built upon the GS paradigm which establishes bidirectional associations between 2D semantic features and 3D feature fields, enabling end-to- end optimization for multi-view consistent feature fusion. FHGS preserves the efficiency and explicitness of the gaussian splatting while overcoming the limitations of rasterization- methods designed primarily for RGB reconstruction. Specifically, each gaussian primitive is augmented with con-differentiable semantic features, which represent the frozen pre-trained features within the 3D space and directly supervised by ground-trul truth feature maps to enforce semantic consistency across views. To better achieve multi-view feature consistency under efficient optimization while preserving isotropic representations within the feature field, we propose a dual-driven mechanism inspired by of External Potential Field Driving and Internal Feature Clustering Driving, constrains anisotropy to photometric properties merely, while enforcing isotropy in the feature field to support consistent semantic representation.
principles from electric field modeling. This mechanism, composed
Extensive experiments of benchmark datasets demonstrate the proposed FHGS not only enhances semantic fusion quality but also improves geometric reconstruction accuracy and noise robustness through naature-driven regularization effects. The main contributions of this work are follows:

---

General feature fusion architecture: We propose a GS-based feature field fusion framework capable of integrating 2D semantic features extracted from large-scale pre-trained models (e.g., SAM [I], CLIP [12) enabling unified optimization from low-level geometry to high-level semanties.
Integration of non-differentiable features into GS framework: We pioneer about integrating of frozen features into the differentiable gaussian splatting methods, which fundamentally resolves the inherent contradiction between the anisotropic nature of gaussian primitives and the isotropic requirements of semantic features.
Physics-inspires dual-drive mechanism: Inspired from electric field modeling, we design joint optimization strategy combining external potential field driving and internal feature clustering driving, characterized by intuitive logic, computational efficiency, and strong interpretability. Additionally, the metric based on this mechanism, named FE, is proposed to evaluate the global consistency of features.
Performance superiority: Compared with other 3DGS feature fusion frameworks on bench- mark datasets, our FHGS achieves state-of-the-an fusion performance and enhances geo- metric reconstruction accuracy, while maintaining a significantly lower training time than our baseline. Therefore, our framework lays the foundation for future extensions toward real-time reconstruction and VLN applications.
2 Related Work
2.1 Novel View Synthesis
Neural Radiance Fields 3 models a continuous 3D scene representation through an implicit radiance field and differentiable volume rendering equation supervised by 2D images. The core of NeRF is that it leverages a multilayer perceptron (MLP) to map spatial positions and viewing directions to color and density values, enabling novel view synthesis with high-quality via ray integration. Subsequent works such as Mip-NeRF 13,, Instant-NGP [4), and Mip-NeRF 360 [15] further improve anti-aliasing, training speed, and scalability to large-scale unbounded scenes. However, the nature of implicit representation of NeRF- based methods [13] [1] [1] requires dense sampling and complex network inference, leading to low training and rendering efficiency that limits its applicability in real-time scenarios.
To overcome the limitations of implicit representations, 3D Gaussian Splatting (3DGS) 4 introduces an explicit scene representation by decomposing the 3D environment into set of explicit, anisotropic gaussian primitives. Combined with the implicit pipeline, this formulation enables efficient training and rendering. Moreover, the anisotropic view-dependent appearance further represented using spherical harmonics, conditioned on the spatial and radiometric properties of each primitive. Com- pared to NeRF, 3DGS eliminates the need for neural networks in the rendering pipeline, significantly improving memory efficiency and real-time performance while maintaining high-fidelity recon- struction. Extending this idca, 2D Gaussian Splatting (2DGS) [66] enhances multi-view geometric consistency by anchoring gaussian primitives to the image plane while enforcing depth consistency constraints. Despite these advances, existing GS-based frameworks remain primarily focused on geometry reconstruction, without addressing Our method FHGS introduces high-level semantic priors from SAM [T]. enabling structure-aware guidance during reconstruction. Different from conventional GS methods that rely purely on dif- ferentiable photometric cues, our FHGS leverages non-differentiable, high-dimensional semantic information to guide the optimization of semantic-aware structural distributions, resulting more precise and robust reconstructions, especially in challenging regions where appearance cues alone are insufficient.
challenge of the utilization of semantic features.
2.2 Implicit Feature Fusion
Integrating semantic information or learned features into baset scene representations is a well- established strategy, extensively explored in the NeRF-based works [17 18
19
20
21 22 and now
migrating to gaussian splatting. Recent attempts to incorporate features into 2D/3D gaussian splatting can be clustered into three categories. Mask fusion approaches exemplified by GaussianCut (3 and
Gaussian Grouping (24] provided 2D masks onto the gaussian primitives set and employ graph-cut
3

---

or low-dimensional identity embeddings for partitioning. While this method is straightforward for interactive 2D editing. the resulting correspondence no longer perceptually obvious in 3D space, and these method still depends on extensive manual annotation, which fails to capture any high- dimensional semantic features. External fusion schemes such as SAGA (25], Semantic Gaussians [26] and OmniSeg31 [27] distilled 2DD features into 3D space through an auxiliary neural network or contrastive learning, thereby enriching semantic information at the expense of additional parameters, prolonged training, and deviation from the concise design philosophy of gaussian splatting. Feature fusion techniques including Feature 3DGS [T] and LangSplat [28] learned embeddings with individ- ual primitive that semantics render with color. However, these embeddings overwrite reshape the high-ding digh-dimensional tensors supplied by large segmentation models, erasing the self- self-attention struc- ture and class relationships encoded therein and often introducing substantial noise that degrades segmentation quality. As consequence, subsequent reasoning is confined to image space rather than the gaussian primitives domain.
Therefore, we model cross- view semantic coherence as a thhysccs-inspired potential-fiele optimization that relocates gaussian primitives while preserving their original feature vectors. The pipeline of our FHGS is self-supervised and globally consistent, retains the full high-dimensional semantic tensor for downstream tasks such as segmentation, detection and multi-modal prompting, and maintains the real-time rendering performance fundamental to gaussian splatting.
Methodology
The proposed FHGS addresses the semantic distortion and efficiency bottlenecks caused by the conflict between the anisotropic rendering mechanism gaussian splatting and the isotropic require- ments of high-level semantic features. There three core components of FHGS, which will be successively illustrated this section: (1) A general-
feature fusion architecture that supports the integration of multi-view features. (2) A GS framework enhanced with non-differentiable features, enabling the incorporation of high-dimensional semantic priors. (3) A dual-driven feature fusion mechanism inspired by physical modeling, which guides the feature optimization process using both geometric and semantic consistency cues.
3.1 General Feature Fusion Architecture
The pipeline of the General Feature Fusion Architecture is demonstrated in Fig. 2 The Structure from Motion (SFM) process reconstructs a sparse 3D point cloud PC via Bundle Adjustment (BA) firstly. To accelerate the correspondence of 3D point cloud and 2D feature, construct spatial hash table that indexes the projections of each 3D point pc; across visible views M. Subsequently, pre-trained model of segmentation is used to generate 2D ground-truth feature maps Fy,, Given a 3D point pc;, its corresponding pixel n in a randomly selected view m E M is retrieved via the spatial hash table, and the semantic feature f, F is sampled accordingly. Each gaussian primitive is
Pre- Train
Operation Flow
Gredieet Flow
Hash
(e)
SFM
Adaptive Density Control
Initialization
P.4.J
Rasterization Framwork
Projection
Figure 2: Pipeline of the General Feature Fusion Architecture
initialized from point pc; and inherits its geometric parameters, appearance attributes, and frozen semantic feature. Specifically, a primitive is anchored at a center position p. and oriented by two orthogonal tangent directions t, and t,, with the normal vector defined t. t, x t,.. These directions form the rotation matrix R. = [tw: tuy tyo] E 3R333 The spatial extent on the tangent plane is described by planar scale vector S, (5)) For appearance modeling, each primitive carries

---

an RGB color C; Rd and an opacity scalar op; E [0, 1]. Additionally, a frozen semantic feature vector f. is assigned to each primitive, extracted via non-differentiable image embedding. We write the complete representation of each gaussian primitive as:
8, = Pr, R. S Opt, C₁2 f.f. }.
(1)
The other symbols follow the notation of conventional 3DGS. Any point (u, u) in the tangent plane is mapped world space by:
P(u, v) = pt + sut, + s,t,u H( U,11,,
(2)
with the homogeneous matrix H E Rdx14 factories translation, rotation and scale. Given local coordinates = (u, v),, the unnormalized density is G(u) exp(- =سtك²² Then, let x = (x, y) be pixel and define u(x) as the unique point in the splat's tangent plane whose homogeneous coordinates satisfy:
(oz, yz, 1) WP(u,)) WH(u,
where W R+x4 is the world-to-camest transformation matrix, and denotes the depth. During the rasterization process in the GS framework, primitives that intersect with the ray emitted from pixel are identified. Specifically, the N primitives covered by ray I are sorted by their rendering depth, with index = and = N assigned to the farthest and the nearest, respectively. The final color can be computed
c(x)
C wi
==1
The weight uy = =,;;; the dynamic differentiable parameter, while a; = op:9i(u(x)) characterizes the intrinsic properties of gaussian primitives, and T; - [1 a,) encodes their transmittance. During the backward propagation, gradients of w, propagate through the chain rule to drive the opti- mization of the geometric parameters of gaussian primitives, thereby enhancing scene reconstruction quality. As the pivotal variable linking geometry and the differentiable rastcrization, w; directly drives both reconstruction accuracy and rendering efficiency.
3.2 Non- Non-Differentiable Features Fusion Mechanism
FHGS integrates a non-differentiable feature driving (NDFD) (orange are pathway in Fig. 3 with the original GS framework. During the forward process, FHGS directly utilizes Fy; compute the feature loss Leat based on the feature f and contribution weights w,. It is worth noting that the forward process does not require prior feature rendering, which further reduce the computational costs. In the backward process, although the feature f; of each gaussian primitive is non-differentiable, the gradient of feat still propagate through wt optimize { p;: R,, S,, op; }. implicitly guiding gaussian primitives toward feature-consistent regions.
NDFD
Feature Gradient Tiles
Losst
Fii (n)
RGB Rasterizalion
fi m4
Backward
ui
Render
FHGS
Ligh
DRF
RGB Gradient Ties
Fealure GroundTrouth
Non-differentisble Feature Driving (NDFD)
Embeddiig
Differentiable Rastrization Framwork (DRF)
Figure 3: Schematic representation of the mechanisms of FHGS: NDFD and DRF
Compared the differentiable rasterization framework (DRF) the conventional GS methods (green are pathway in Fig, 3) the non-differentiaabl branch climinates the need for feature rendering
5

---

during the forward process, enabling direct loss computation while preserving the efficiency of GS framework. This design brings the following characteristics: the anisotropic color rendering remains dedicated to illumination and shadow modeling, while the multi-view consistency of non- differentiable features is achieved through w-driven distribution optimization, thereby avoiding direct conflicts between the rasterization anisotropic mechanism and the isotropic requirements of semantic features. The detailed pseudo code NDFD is given in the Appendix.
3.3 Physics-Inspired Dual-Drive Mechanism
Inspired by principles from an intuitive analogy in physical field theory, we model the Fm within gt
the rasterization as a feature field in homogeneous space defined in the Eq. we consider it as an "electric field". More concretely, as illustrated in the Fig. we treat the ray emitted from pixel an electric field line, and define its semantic property the ground-truth feature fz= F (n) sampled from the 2D feature map. The gaussian primitives are conceptualized as discrete "charges" carrying intrinsic features f.. The feature loss feat is then formulated as the potential energy loss in this electric field analogy. During the backward process, gradients drive the spatial optimization of gaussian primitives, analogous to the motion of charges under electric field forces toward regions of lower potential energy.
External Potential Field Constraint: Following the logic of NDFD, we compute the cumulative similarity between the features f; intersecting with ray and the ground-truth feature fgt, constructing similarity loss during the forward process:
Let
w
(5)
To eliminate the inherent contradiction between gaussian primitives and ground-truth semantics in the feature space, FHGS introduces a similarity- activation function o; -+=*(--xy, where cos (f, fgt). This sigmoid function maps feature similarity into a polarity-like response. analogous the binary behavior of electric charges. More detailed explanation of sigmoid function are given in the Appendix.
F
F
fz-3
Fy
fge
C2
Figure 4: The illustration of proposed Dual-Drive Mechanism: The color of each gaussian primitive and ray represents their feature properties, and the transparency represents the magnitude of the weight w/ of the primitive on the ray. fy-3 exhibits similarity to posterior accumulated values F,-1, which is the value of the cluster C2 only constrained by Lgti fN-4 44 represents the inter-cluster noise points of C and C2 suppressed by Let and Lef: f2 corresponds to the internal noise points from cluster C1 where both Legt and Lef effectively optimize the distribution of Ci.
Internal Clustering Driving: In order to suppress noise, enhance semantic coherence, and quantify the semantic feature entropy at pixel n, we simplify the bidirectional traversal of feature similarity between gaussian primitives during the rasterization process
N
Les
w.W;y (1 f()
(6)
N
o, (W;-1 F,-1 -f)

---

The detailed derivation process of Lef can be found in the Appendix. Since both f, and f are normalized, cos f,) simplifies to f. f,. We can obtain the cumulative weight W,, Σi=1 w; and cumulative feature Fn = Ei=1 w,f, along the ray from far near. In addition, each current feature f is compared only with the cumulative feature F This avoids spurious repulsion across unrelated objects and reduces complexity from ( to O(N). Furthermore, the cumulative weight W encodes the rendering contributions of farther gaussian primitives, implicitly modeling depth hierarchy. More specifically. the similarity activation function σ; suppresses interference from background clutter, preventing incorrect contributions to foreground semantic clusters (e.g., C1 and in the Fig. 4. This mechanism achieves local semantic coherence by anchoring primitives that are semantically consistent with Ege (e.g., fi), while repelling dissimilar ones, thereby reducing feature conflicts and reinforeing cluster purity. effectively suppresses internal noise (e.g.. f2 in the Fig. + and eliminates irrelevant outliers in space (c.g., C2 and f 3 in the Fig. resulting in cleaner and more compact semantic regions.
The aforementioned two driving methods, together with Ligb, jointly constrain the semantic fusion process of 3D scenes. The external potential field driving ensures semantic consistency across views, while the internal clustering suppresses outlier noise and enhances intra-cluster coherence. Moreover, the nntetnalclustering term L refines the tine-graine details captured by Lgt and accelerates its convergence. Two external semantic guidance and internal clustering regularization, respectively. Finally, we define the overall loss J. as:
λ and A2 are manually selected to balance the contribution of
L Ligh + An + d2Les
Under the NDFD mechanism, gradient with respect to wy not only influence the geometry and appearance of local primitive but also affect the spatial distribution of subsequent gaussian primitives in the backward traversal. The gradient can be obtained by:
OLef (W - Fx-1 + Owk
N
Σ
o,w;(1 - f
The symmetry between the forward and backward passes allows cumulative terms computed during the forward traversal to be directly reused in gradient calculations (see Appendix), eliminating redundant passes and preserving the O(N) complexity of both processes.
ss
Experiments
We implement FHGS within 2DGS-based framework, deploying tailor-made CUDA kernels accelerate the proposed feature-fusion operations. We use the image embedding of SAM 11 as input to the feature. The original 2DGS renderer is retained to export depth-distortion maps, depth maps, normal maps, and mesh reconstructions, which serve as the inputs to our quantitative and qualitative evaluations. In the sigmoid activation function, the similarity threshold and slope are empirically fixed to x = 0.5 and k = 20, respectively, ensuring stable binarization of the naature-machiin score that governs the polarity of gaussian primitive. For benchmarking, we adopt Feature3DGS [10] as the baseline. Following its protocol, we report the L, feature loss FL1 (lower values indicating higher feature similarity) under the same rendering pipeline, where smaller FL.1 values signify better feature fusion. Cross-view consistency is further assessed with the ground-trute entropy metric Lg (Eq, 53; lower Legt scores indicate tighter multi-view alignment. To ensure fair comparisons, all experiments are executed on workstation equipped with single NVIDIA GeForce RTX 4090 (24 GB) and an AMD Ryzen 9950X (16 cores). In addition, we use identical feature-extraction pipelines together with the default 2DGS optimizer settings (learning rate, iteration count, batch size) for both the baseline and our method, thereby climinating performance biases due to hyperparameter tuning or ceatuee-generation differencee.
4.1 Comparative experiment
To verify the generalization and robustness of our method, we conduct systematic experiments on a range of public datasets covering both indoor and outdoor environments. For indoor evaluations,
7

---

Table 1: Quantitative results comparison on indoor scenes
DTU-24
29
DTU-37
29
MN360-Kitchen
hk
Method
PSNR1
FE]
FL14
Timel
PSNR1
FE].
FLL.
Timel
PSNR1
FE]
FL1.
Timel
2DGS
30.1
1.35
0.61
6.1m
30.5
1.31
0.52
6.3m
30.2
1.32
0.79
6.5m
Feature3DGS
31.5
0.52
0.24
82.2m
31.1
0.88
0.31
73.2m
31.7
0.63
0.31
113.2m
FIGGS (ours)
30,9
0.15
0.22
5.2m
30.8
0.21
0.18
5.7m
30.8
0.23
0.21
5.1m
the results shown in Table while outdoor evaluations are performed on Mip-NeRF 360 (Garden, Stump) and Tanks and Temples (Tn'I Caterpillar) [30], the results shown in the Table 2 All input images
evaluate method DTU (scans 24, 37) 29 and Mip-NeRF 360 (Kitchen) [15]
are uniformly downsampled to a maximum side length of 1,000 pixels to balance computational efficiency and reconstruction accuracy. Sparse point clouds initialized with COLMAP [31] used for SIM, with fixed iteration count of 10,000 to ensure optimization consistency. During testing, Feature3DGS [0] failed TnT 30 due to huge utilization of GPU memory.
The experimental results demonstrate that FHGS reduces training time by 15x relative to Fea- ture3DGS, improves performance by 8-10% over standard 2DGS, and maintains real-time rendering at 2 60 FPS. terms of feature fusion quality, FHGS achieves the same performance comparable to Feature3DGS in the FL.1 metrics, validating its effectiveness in feature similarity measurement. FHGS also exhibits superior FE metrics (lower values denote stronger cross-view consistency), highlighting its advantage in semantic coherence across viewpoints.
Table 2: Quantitative results comparison on outdoor scenes
COLMAP 31
MN360-Gurden
15
TnT-Cuterpillar
Method
PSNR1
FE]
FLL
Time.
PSNR
FE]
FLI.
Time.
PSNR
FEІ
FL1
Time.
2DGS
27.4
1.73
0.83
10.16m
31.3
1.67
0.75
6.3m
26.8
1.72
0.76
5.2m
Feature3DGS
28.2
0.55
0.42
181m
31.6
0.65
0.33
155.4m
FIGGS (ours)
26.5
0.25
0.24
7.8m
30.6
0.25
0.18
6.1m
26.6
0.21
0.41
5.2m
BGI anoued trath
Oaniliiiii
Feature grous trik
2DUSSaommal)
Oarvitacuall
Figure Qualitative results to compare our FHGS with Feature3DGS 10 2DGS [6 in normal map.
feature map and
To visually assess fusion quality, we map channels 15, 28 and 31 of the image embedding features to RGB for rendering. The visualization comparison results in Fig.55urrther demonstrates the superiority of our FHGS which produces uniform feature distributions with minimal noise, smooth semantic transitions, and clear boundaries. For geometric reconstruction, our method effectively suppresses noise and drives geometric structures to converge toward thin planar surfaces, ultimately achieving high-precision surface reconstruction comparable to MVS. Furthermore, the quantitative results show

---

that our method achieves higher PSNR values than the baseline in indoor scenes, but lower PSNR in outdoor scenarios. This improvement arises because low-semantic features such as soil and shadow regions are effectively filtered out during reconstruction (See Appendix).
Our training is faster and uses less GPU memory in Table These results conclusively demonstrate consistency in 3D scene representations
that FHGS significantly enhances semantic-geomet while preserving real-time rendering efficiency through the proposed novel fusion mechanism and optimization strategy. More results of experiments can be found in the Appendix.
Table 3:
Quantitative results between FHGS, 3DGS, 2DGS and Feature3DGS on the DTU 29 report chamfer distance, PSNR (training-sel view), reconstruction time, model size and point number.
Methods
CDf
PSNRT
Time!
PNL
MB (Storage)
3DGS
1.96
35.76
11.2m
532k
113
2DGS
0.83
33.42
5.5m
342k
52
Feature3DGS
1.85
35.25
>24h
642k
745
FHGS (ours)
0.75
34.21
4.8m
196k
183
4.2
Ablation Study
The ablation study is conducted on the scan24 of DTU dataset [29] with 10,000 training iterations to investigate the effects of the loss functions L.g and Tey on feature fusion, geometric reconstruction, and optimization efficiency (illustrated in T))le Fig. (a) illustrates the result of image embedding from SAM 11] The experimental results indicate that these two loss terms serve complementary roles: As illustrated in the Fig. (d), when Lgt and Lef are both disabled, and visualizing the feature through the default rendering logic, the resulting feature map diverges markedly from the ground truth and appears cluttered. When only Lef is removed, although the optimization proceeds faster, the model suffers from semantic contamination and overfitting at the surface level. As shown the Fig, (c), numerous valid gaussian primitives are incorreetly discarded, leading to excessive transparency in the reconstructed geometry and severe degradation in reconstruction quality. When both Lyt and Lef jointly applied, the framework achieves optimal balance: the feature consistency metric FE improves, geometric structures converge toward thin and planar forms, and convergence speed increases. Fig. (b) has shown that under this configuration, the distribution of the feature field is uniform and dense, semantic boundaries are sharp and well-defined, and the reconstructed surfaces retain detailed geometric information. These findings validate the effectiveness of the dual-loss collaborative optimization strategy.
(a) Feature ground truth
(b) FHGS (ours)
(c) FHGS (w'o Ley)
(d) 2DGS (baseline)
Figure 6: Ablation study on the scan24 of DTU dataset 29
Table 4: Quantitative analysis of ablation experiments on DTU scan24
Methods
PSNR1
FE]
FL1
Time.
PN.
FHGS (ours)
30.9
0.15
0.16
5.2m
214k
FHGS (w/o Lef)
27.2
0.10
0.21
4.3m
217k
2DGS (baseline)
30.1
1.35
0.46
6.1m
329k

---

Conclusion and Discussion
We introduce Gaussian splatting based framework named FHGS, which incorporates non-differentiable feature-driven regularization term to enforce multi-view semantic consistency. FHGS markedly boosts multi-view feature alignment and geometric reconstruction quality while maintaining real-time performance, as demonstrated by extensive experiments on diverse indoor and outdoor datasets. While our FHGS successfully achieves multi-view consistent and accurate geometric reconstruction, still has some limitations: our methods remains sensitive to the manually tuned similarity activation aimilartt--cctivation parameters lambda and k; its hash-table and cumulative-weighte buffers incur considerable GPU memory in large-scale scenes. In future work, plan explore adaptive pa- rameter learning strategies to reduce dependence on manual tuning, and to develop memory-efiicien and compact representations to enhance scalability in large-scale environments.
Acknowledgments and Disclosure of Funding
This study was supported by the InnoHK initiative of the Innovation and Technology Commission of the Hong Kong Special Administrative Region Government via the Hong Kong Centre for Logistics Robotics, and in part by the Research Grants Council of Hong Kong SAR under Grants 14206821, 14217922 and 14209623.
References
[1] Yao Yao, Zixin Luo, Shiwei Li, Tian Fang, and Long Quan. Mvsnet: Depth inference for unstructured multi-view stereo. European Conference Computer Vision (ECCV), 2018.
[2] Chunran Zheng, Wei Xu, Zuhao Zou, Tong Hua, Chongjian Yuan, Dongjiao He, Bingyang Zhou, Zheng Liu, Jiarong Lin, Fangcheng Zhu, Yunfan Ren, Rong Wang, Fanle Meng, and Fu Zhang. Fast-livo2: Fast, direct lidar-inertial-visuas odometry. IEEE Transuctions on Robotics, 41:
326-346, 2025. doi: 10. (009/TR..002.33555
[3] Ben Mildenhall, Pratul Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. In ECCV, 2020.
[4] Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Transactions Graphics, 42(4), July 2023.
[5] Shi-Sheng Huang, Haoxiang Chen, Jiahui Huang, Hongbo Fu, and Shi-Min Hu. Real-time globally consistent 3d reconstruction with semantic priors. IEEE Transactions on Visualization and Computer Graphics, 19(4):1977-199) 2023. doi: 10. 10.10099TVCG.22213333911
[6] Alexey Dosovitskiy,, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations, 2021.
[7] Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 15988, 2022. doi: 10.1 10.1109/CVPS52288.2222 01553.
[8] Yawar Siddiqui, Lorenzo Porzi, Samuel Rota Bulò, Norman Müller, Matthias Nießner, Angela Dai, and Peter Kontschieder. Panoptic lifting for 3d scene understanding with neural fields. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 9043-9052. June 2023.
[9] Shuaifeng Zhi, Tristan Laidlow, Stefan I Leutenegger, and Andrew J Davison. In-place scene labelling and understanding with implicit scene representation. Proceedings of the IEFF/VE International Conference Computer Vision, pages 15838-15847, 2021.
10

---

[10] Shijie Zhou, Haoran Chang, Sicheng Jiang, Zhiwen Fan, Zehao Zhu, Dejia Xu, Pradyumna Chari, Suya You, Zhangyang Wang, and Achuta Kadambi. Feature 3dgs: Supercharging 3d gaussian splatting to enable distilled feature fields. In Proceedings of the IEEF/CVE Conference on Computer Vision and Pattern Recognition, pages 21676-21688 2024.
[11] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan-Yen Lo, Piotr Dollár, and Ross Girshick. Segment anything. arXiv:2304.026643 2023.
[12] Alec Radford, Jong Wook Kim. Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Proceed- ings of the 38th International Conference on Machine Learning. volume 139 of Proceedings of Machine Learning Research, pages 8748 8763. PMLR, 18-24 24 Jul 2021.
[13] Jonathan T. Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman, Ricardo Martin-Buualla, and Pratul P. Srinivasan. Mip-nerf: multiscale representation for anti-aliasing neural radiance fields. ICCV, 2021.
[14] Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphics primitives with a multiresolution hash encoding. ACM Trans. Graph., 11(4):102:11102:15,, July 2022. doi: 10. 00.1145/32822333530011
[15] Jonathan T. Barron, Ben Mildenhall, Dor Verbin, Pratul P. Srinivasan, and Peter Hedman. Mip-nerf 360: Unbounded anti-aliased neural radiance fields. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5460-5469, 2022. doi: 10.1109/
SPPP52688.2022.0000
[16] Binbin Huang, Zehao Yu, Anpei Chen, Andreas Geiger, and Shenghua Gao. 2d gaussian splatting for geometrically accurate radiance fields. In SIGGRAPH 2024 Conference Papers. Association for Computing Machinery, 2024. doi: 10. 00.114/9641519.3657477
[17] Zhiwen Fan, Peihao Wang, Yifan Jiang, Xinyu Gong, Dejia Xu, and Zhangyang Wang. NeRF- SOS: Any-view self-supervised object segmentation on complex scenes. In The Eleventh International Conference on Learning Representations, 2023.
[18] Justin Kerr, Chung Min Kim, Ken Goldberg, Angjoo Kanazawa, and Matthew Tancik. Lerf: Language embedded radiance fields. In International Conference on Computer Vision (ICCV), 2023.
[19] Sosuke Kobayashi, Eicchi Matsumoto, and Vincent Sitzmann. Decomposing nerf for editing via feature field distillation. In Advances in Neural Information Processing Systems, volume 35, 2022.
[20] Vadim Tschernezki, Iro Laina, Diane Larlus, and Andrea Vedaldi. Neural feature fusion fields: 3D distillation of self-supervised 2D image representations. In Proceedings of the International Conference on 31) Vision (3DV), 2022.
[21] Shuaifeng Zhi, Tristan Laidlow, Stefan Leutenegger, and Andrew J. Davison. In-place scene labelling and understanding with implicit scene representation. In ICCV, 2021.
[22] Yawar Siddiqui, Lorenzo Porzi, Samuel Rota Bulø, Norman Müller, Matthias NieBner, Angela Dai, and Peter Kontschieder. Panoptic lifting for 3d scene understanding with neural fields. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 9043-9052, June 2023.
[23] Umangi Jain, Ashkan Mirzaei, and Igor Gilitschenski. Gaussiancut: Interactive segmentation via graph cut for 3d gaussian splatting. In The eiirth- Annual Conference on Neural Information Processing Systems, 2024.
[24] Mingqiao Ye, Martin Danelljan, Fisher Yu, and Lei Ke. Gaussian grouping: Segment and edit anything in 3d scenes. In ECCV, 2024.

---

[25] Jiazhong Cen, Jiemin Fang, Chen Yang, Lingxi Xie. Xiaopeng Zhang, Wei Shen, and Qi Tian. SAGA: Segment any 3d gaussians. Proceedings of the AAAI Conference on Artificial Intelli- gence, (971))117111979 2025. doi: 10.1609/aaai.v399..2 v3912.32193.
[26] Jun Guo, Xiaojian Ma. Yue Fan, Huaping Liu, and Qing Li. Semantic gaussians: Open- vocabulary scene understanding with 3d gaussian splatting, 2024.
[27] Haiyang Ying, Yixuan Yin, Jinzhi Zhang, Fan Wang. Tao Yu, Ruqi Huang, and Lu Fang. Omniseg3:: Omniversal 3d segmentation via hierarchical contrastive learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pages 20612-20622, June 2024.
[28] Minghan Qin, Wanhua Li, Jiawei Zhou, Haoqian Wang, and Hanspeter Plister. Langsplat: 3d language gaussian splatting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 20051-20060 June 2024.
[29] Rasmus Jensen, Anders Dahl, George Vogiatzis. Engil Tola, and Henrik Aanæs. Large scale multi-view stereopsis evaluation. In 2014 IEEE Conference Computer Vision and Pattern Recognition, pages 406-413, 2014.
doi: 10. 10.V.
[30] Arno Knapitsch, Jaesik Park, Qian-Yi Zhou, and Vladlen Koltun. Tanks and temples: benchmarking large-scale scene reconstruction. 36(4), 2017. ISSN 0730-0301. doi: 0011145/3072959.0733599
[31] Johannes L. Schönberger and Jan-Michael Frahm. Structurefrrom-motion revisited. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 4104-4113, 2016. doi: 10. 10.1109/VVPR..20...445
12

---

NeurIPS Paper Checklist
The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No], or [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justifiicton right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation.
While "[Yes] " is generally preferable to "[No] ", it is perfectly acceptable to answer "[No] provided proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] or "[NA] is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification: The main claims made in our abstract and introduction correctly reflect the contribution and scope our paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]
13

---

Justification:
The paper comprchensively discuss the limitation in the section of Conclusion.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "I "Limitations" section in their paper.
The paper should point any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model sele--pecfiiatinn. asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: All the formulas in the paper are numbered and cross-referenced, the detailed proofs of core theorems given in the section of Appendix.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We describe the two architectures in combination with the schematic diagrams in the paper. We discuss and explain our proposed algorithm through experiments on variety of datasets.
14

---

Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g.. a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open- open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes)
Justification: We have prepared all the relevant code and ensured that they can be easily reproduced. If our paper is accepted, we will immediately open source all the data and code.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ ptaips.cc//ublic//uuu
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
15

---

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We have specified all the training and test details necessary to understand the results in the beginning of EXPERIMENTS.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by enror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: We have provided the sufficient information about the computer sources used in experiments in the section of EXPERIMENTS.
Guidelines:
16

---

The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics htttps://neuripge: htcc/ppbbic//thicssu..eeelllllll
Answer: [Yes]
Justification: All the research conducted in our paper conform the NeurlPS Code of Ethics in every respect.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations in their jurisdiction..
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [No]
Justification:
[NA]
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications,
authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
17

---

Answer: [No]
Justification: [NA]
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All assets mentioned in the article have been correctly cited, explicitly men- tioned and respected, and comply with all licensing and usage terms.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URI
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, pwpprswithhaeecoc thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-pane se-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We have provided detailed documentation for all proposed assets, and once the article is accepted, we will open source together with all the assets.
Guidelines:
The answer NA means that the paper does release assets.
Researchers should communicate the details the dataset/codeemoodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
18

---

14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [No]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/teview based the requirements of your country institution) were obtained?
Answer: [No]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMss if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [No]
Justification: [NA]
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our L.M for what should or should not be described.
19

---

A Detailed Explanation of Non-Differentiable Feature Driving (NDFD)
Algorithm 1 General Feature Fusion and Densification Framework
gt <- GFTFEATUR:FFOMSAM gt
Features Positions & Hash
(p, H) < GETPOINTFROMSFFM gt)
f f POINTSFEATUREFUSSINN,
H)
b Point Features
(R. cp) + INITATTRIBUTES()
> Rotations, Scales, Color, Opacity
b Iteration Counter
while not converged do Fm
(m, gen gt <- SAMPLETRAININGVIEN))
Camera, Image, Feature Rasterization
1 < Dp R. S. e,
Ligh + LOSS Th 1
> Photometric Loss
(Leg, Les) NDFD(p, R, S, f, Fm m)
> Semantic Loss
L+ Lighh + Let + Lef
Total Loss
S. t ADAM( IRREENEMENTIERATIONN then DENSIFICATION(P: R, S, o,
Update
Adaptive Density
end
i f- i+1
end while
Details of the Rasterizater: Our implementation builds directly on the GPU rasterizer proposed in 3D Gaussian Splatting. Following that design, the image plane of size w X h is partitioned into 16 16 px tiles. Each gaussian primitive that overlaps tile is duplicated for that tile and assigned 64-bit key whose lower 32 bits encode depth and upper bits encode the tile index. A single parallel radix sort on these keys resolves global depth order and produces a compact, per-tile, depth-sorted list of instances; a second pass identifies the start-end range for each tile (see CULLIASSSIAN, DUPLICATEWITIKEE and SORTB YEYS 3DGS). This eliminates sequential primitive traversal and maximizes GPU utilization.
Algorithm Non- Non-Differentiable Feature Driving Mechanism
function NDFD R.S, op, f, Fm (i,m)) m)
+- HOMOGENIZATION(mI
Camera Homogenization b Screen-space Gaussians
g (u(xx)) + 2DSCRENGAUSSIANSS/ R, S,
T +- CREATETILES(()
b Tile Grid
(I, K) + -DUPLICTTEITIIKESS/A
> Indices & Keys Global Sort
SORTBYKEYS(K,I)
T, + INENT K)
> Tile Ranges
Let + 0, Le/+0 +0
Initilize Loss Buffers
for all t do
for all i do
t- GETTILERANGE(T,.C
Index Range in K
for all j r do
j SIGMOID (i),
Polarity Response
W/ + WEIGHTCALC( (gj. op;)
Opacity-weighted weighted Area
Opacity-
T += EPFC(U;, {j
External Potential
Lef
(cD(w,,Jj.0)
Internal Clustering
end for end for
end for
return (Lez, Los) end function
Non- Non-Differentiable Feature Driving (NDNF): Alg. augments the aforementioned rasterizer with feature-centric branch that runs entirely on the sorted gaussian primitives lists and never invokes
20

---

blending. Given the current view m, camera homogenization first projects gaussian primitives means into screen space, after which key generation and radix sorting produce per-tile ranges. For every pixel i in a tile I, we then traverse the corresponding range in front-to-hack order. A sigmoid activation o = SIGMOID(F") gr (i), fj) converts the cosine similarity between the frozen feature S, of the j-th gaussian primitives and the ground- -hruth embedding Fg into a charge-like polarity. The raster weight Wj, which combines projected area and opacity exactly as in blending, is accumulated only by this feature branch. Then, two loss terms are computed: the external-ootential loss Lye attracts -y-weighted features toward ge whereas the internal-clustericn cumulative-feature rule to penalize incoherent neighbors. These loss buffers are initialized once per frame and updated atomically in the innermost loop, During back
loss Lef applies the
intermediate feature image is rendered. back-propagattinn propagation, gradients propagate solely through the weights wy, which reuse the same cumulative prefix employed for a-blending in the forward traversal, thereby retaining the (NN) complexity of the original rasterizer.
A.1 Derivation of the feature similarity
Internal clustering loss Lay: For a given pixel p, let {(w, fi) be the set of gaussian primitive whose screen-space footprints cover that pixel, where w/ is the weight and fi E Rd is the frozen semantic feature of the th primitive. The internal-clusternng loss:
Lef Σ>
w; wy (1-cos(fi fz))
computes the entropy of the local feature distribution by accumulating the weighted cosine dissim- ilarity between every pair of primitives. The process of minimizing Lef pushes feature vectors of neighboring primitive align, suppresses noisy outliers, and tightens semantic coherence within the pixel neighborhood. And simultaneously. this process allows primitives belonging to different objects to repel each other through their low cosine similarity.
We further convert to an ((N) backward traversal by noting that feature vectors are normalized, therefore cos(fi, f2) = fi- ʃz. We can rearranged the representation of Lef as:
Lef
Ox W. wy cos fz))
i-1
Σ
oi Wi
Σ Wi -
Σ
Wi Si fi
j=1
(8)
i-1 Σ
oaw;
Σ W3
wifi ft
5=1
j=1
Ti wi (W;-1 F-11 f()
where the cumulative weight W,- and cumulative feature F:-1 are updated one time in each step during the front-to-back blend. The final form Lef retains the physical meaning of pairwise semantic intractioo-repulsionn evaluates in O(N) time. The cumulative values of WN-1 and FN-1 are recorded.
A.2 Calculation process of the gradients
As derived in the main text, we obtain the partial derivatives:
O = σ (Wk-1 - Fs-1 -a1 + Owk
N
Σ k=
o,w; (1 f, fe)
(9)
In the forward pass the gaussian primitive processed descending depth order from the farthest to the nearest with respect to the camera. The backward pass visits the same primitive in the reverse
21

---

order. Because the index k is defined with respect to the forward ordering. we re- re-index the backward forkard-backward of the internal-clustering loss with respect to the weight of the current primitive can be rewritten as:
traversal by a new counter q 1.. ,....,N, .N. Exploiting this forward backward symmetry, the gradient
Thes (o We) (FN Fg) fg) Owg
Jw (1
fq)
(10)
Based on the aiT 0; op,9i T, -1111 Og}, we can obtain:
at
Dwk
ÖEgt
1
a
wi
(11)
8or
Owk
dak
Owk.
Ot
Owi
x=
Analogously to the equation above, we can also obtain:
aLyt
aL
Owg dag
aL
Lgt
Tg
wi
(12)
Oog
Ow,
owl
04g
Owi
i=0
B Comprehensive Results of Experiments
conduct a detailed comparison between our method and Feature3DGS on the DTU indoor dataset. As shown in the Fig. 7 our method yields more uniform feature distributions and sharper boundaries. Moreover, it effectively suppresses background clutter, which remains prominent in Feature3DGS. The enhanced clarity and selectivity of our features also benefit downstream tasks such as segmentation and reconstruction. These observations highlight the strength of our feature driving mechanism in promoting structural coherence and semantic focus.
DoS
Figure 7: Qualitative results compare FHGS with Feature3DGS in feature field. The results shown that FHGS achieves better feature extraction with more uniform feature distributions, shaper boundaries and cleaner background.
We further evaluate method against Feature3DGS challenging outdoor scenes from the TnT and MipNeRF360 datasets. As shown in Fig. 8 our method consistently delivers more coherent and spatially uniform feature fields, with significantly clearer object boundaries and effective suppression of background noise. In addition to semantic features, we also visualize the surface normal maps extracted from reconstruction, which exhibit plausible geometric structures and fine-grained surface details. These results demonstrate the robustness of our method under natural lighting, large-scale geometry, and high-frequency textures, confirming its generalization to diverse outdoor environments.
22

---

RGB ground truth
Ours (semnal)
Ours (feature)
FeatureDDGss (leature)
Figure 8: Qualitative comparison on outdoor scenes from TnT and Mip3oR
23


Reviews:
[{"review_id": "NlbK2vfKqe", "reviewer": "Reviewer_omCK", "summary": "This paper proposes a feature fusion framework based on Gaussian Splatting which is able to map 2D features of pretrained models onto 3D scenes while allowing real-time rendering. The key contributions are a universal feature fusion architecture for 2D to 3D feature embedding, a non-differentiable feature fusion mechanism that enables view independent semantic features, and a dual driven optimization strategy that combines external supervision with internal guidance. Experiments show that the method outperforms sota methods in benchmark datasets.", "questions": "1. While the proposed method enhances feature fusion, it also leads to a noticeable drop in PSNR, as observed in Tables 1 and 2. Could the authors provide insight into why improved feature fusion may come at the cost of reconstruction quality, particularly in terms of PSNR?\n2. It would be helpful if the authors could clarify the training losses used for the baselines reported in Tables 1 and 2. For instance, Feature3DGS appears to have feature loss as part of its training objective. If that is the case, do the results suggest that the proposed method outperforms another with explicit feature loss as training objectives?\n3. In addition, I am still not very sure whether feature loss alone is a sufficient metric for evaluating the quality of the fused features. Could the authors consider providing additional quantitative or qualitative results, for example, utilizing SAM decoder on the fused feature map? I am curious whether this would outperform directly applying SAM to the image.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The paper is also clearly written and easy to read.\n2. I think the motivation of this paper is clear and unique. It is interesting to find the existing methods on learning feature fields through 3DGS is facing limitations from conflict between anisotropic RGB fields and isotropic semantic features.\n\nWeaknesses:\nWhile the paper introduces a method that enhances feature fusion, it does not provide sufficient downstream evaluations to fully validate the effectiveness of the learned features, and it lacks investigation into the trade-off with PSNR degradation.", "was_revised": true, "final_justification": "Overall, my questions have been well addressed. I recommend that the authors incorporate these clarifications into the final version of the manuscript to improve the clarity and understanding of the proposed method. I would like to maintain my score and recommend acceptance of this paper.", "rebuttal": "We sincerely thank the reviewer for the insightful comments and constructive suggestions, which have greatly helped us improve our work. \n\n**Response to Q1**:  Just as you suggested, we should indeed dicuss this issue in the paper. We appreciate the reviewer's careful observation regarding PSNR. We would like to clarify that FHGS does not sacrifice rendering quality but actually enhances it in different ways:\n\n*For Indoor Scenes*: \n\n(1) In controlled indoor environments, FHGS demonstrates superior performance across the board. As shown in Table 1 and Table 3, FHGS achieves better PSNR than our baseline 2DGS in indoor scene datasets.\n\n(2) Figure 7 in the appendix demonstrates superior geometric surface constraints and noise reduction compared to 2DGS.\n\n(3) Figure 1 in the main paper shows that our physics-inspired approach effectively removes reverse noise while maintaining better geometric surfaces through 2DGS's normal and depth constraints.\n\n*For Outdoor Scenes*:\n\n(1) Table 2 indeed shows relatively lower PSNR for FHGS compared to baseline 2DGS in outdoor scenes. We believe this is due to unbounded scenes and instabilities in SAM semantic-fields (shadows, dynamic objects). FHGS's multi-view semantic consistency and non-differentiable feature settings may discard unstable information due to insufficient SfM initialization point-cloud density.\n\n(2) As shown in Figure 5 in main paper and Figure 7 in Appendix, this results in holes in feature maps, particularly in ambiguous regions (soil near tree roots, vegetation distant excavators, shadow of grass near the garden). We acknowledge this trade-off - while it affects PSNR metrics, it enables better focus on central objects following self-attention logic, which may benefit downstream applications.\n\nWe extend our sincere gratitude to the reviewer for this insightful comment, which have given us a clearer and renewed understanding of our method. Our FHGS method demonstrates superior PSNR performance in indoor scenes, and its design for outdoor scenes trades a minor decrease in this single metric for a significant gain in semantic robustness and geometric accuracy, as evidenced by our comparative visualization experiments.\n\n**Response to Q2**: We thank the reviewer for raising this important question about the fundamental differences between Feature3DGS and FHGS in terms of their alignment with 3DGS's core philosophy:\n\n*Advantages of our FHGS*: \n\n(1) Computational Efficiency: Direct feature storage and projection is significantly faster than MLP inference during rendering, and it is also faster than our baseline method 2DGS; \n\n(2) Interpretability: Features retain their semantic meaning from SAM's pre-training, unlike learned MLP features;\n\n(3) Better Feature: Feature distribution is excellent which can imply our design for the feature characteristics is even more rational.\n\n(4) GPU-friendly: FHGS exhibits lower memory usage and higher efficiency when training on more complex datasets, leading to better performance. In contrast, Feature3DGS cannot complete training on large-scale, complex datasets.\n\n*3DGS's Explicit Nature*: \n\n(1) Each Gaussian primitive explicitly stores attributes (position, color, opacity, covariance).\n\n(2) The overall method operates without MLP-like hidden layers or implicit functions.\n\n(3) Color rendering follows a differentiable framework using anisotropic spherical harmonics.\n\n(4) Each gaussian primitive's contribution is clear which provides interpretable scene representation. \n\n*From MLP Perspective - Feature3DGS vs FHGS*:\n\n*Feature3DGS* introduces MLPs for feature learning and dimension reduction, adding an MLP implicit layer that we believe diverges from 3DGS's explicit philosophy.\n*FHGS* requires no additional hidden layers or implicit structures, which aligns with the 3DGS nature described above.\n\n*From Loss Design Perspective - Feature3DGS vs FHGS*:\n\nUnlike Feature3DGS which mechanically adapts RGB rendering logic, our FHGS designs a differentiable architecture based on the characteristics of non-differentiable high-dimensional semantic features, with independently designed losses for this architecture. Our FL1 metric, following Feature3DGS's evaluation criteria, demonstrates superiority in feature rendering. Additionally, our FE loss quantifies the entropy of feature consistency confusion in 3D space, showing we also excel from a 3D spatial distribution perspective.\n\n\n**Response to Q3**: We greatly appreciate the reviewer's valuable suggestion, which indeed provided interesting insights. We have conducted the recommended experiment - applying SAM decoder on our fused feature maps.\n\n*Experimental Setup and Findings*: We test on the DTU dataset, using FHGS-rendered feature maps as input to SAM's Image Decoder. The native SAM outputs three segmentation results based on IoU scores. Interestingly, when applying SAM decoder to our FHGS features, we find that:\n\n(1) The first two segmentation results match those of native SAM;\n\n(2) The third (most detailed) segmentation shows some differences - while it successfully segments walls, windows, and individual window panes, it cannot segment individual bricks as native SAM does.\n\n(3) Feature3DGS shows inconsistent performance in Feature-Render results for 2D segmentation across different viewpoints, which means some rendered images are usable while other show poor quality. In contrast, FHGS demonstrates consistent rendering results across all viewpoints, with performance comparable to native SAM. This verify that FHGS provides more reliable feature quality across different viewing angles, which is significant for practical applications.\n\n(4) Unlike Feature3DGS and other Feature-Render approaches that primarily focus on 2D feature rendering quality, our FHGS method emphasizes feature driving in 3D space. Specifically, the native feature data of Gaussian primitives in FHGS exhibits better spatial distribution in 3D space, which is more conducive to subsequent 3D segmentation tasks. This fundamental difference in approach - prioritizing 3D feature coherence over 2D rendering optimization - represents a key conceptual advancement that better aligns with the inherent 3D nature of the scene representation.\n\n*Our Analysis*:\nWe deduce this phenomenon may be attributed to the lower density of SfM-initialized point clouds. The coarser initialization may limit the capture of extremely fine-grained details in our 3D representation.\n\nThis experiment has been enlightening and suggests that while our feature fusion maintains semantic understanding at most scales, there may be a trade-off at the finest detail level. We thank the reviewer for this insightful suggestion, which has helped us better understand the characteristics and limitations of our approach."}, {"review_id": "jak7aK1wT9", "reviewer": "Reviewer_tiJD", "summary": "This paper introduces a novel method with a strong physical motivation for 3D reconstruction. The manuscript is well-written, the figures are high-quality, and the preliminary results are promising. However, the work in its current form would be significantly strengthened by providing more rigorous experimental validation for some key claims and expanding the comparison to include more recent state-of-the-art methods. The main concerns are the unsubstantiated claims regarding performance degradation due to feature conflicts and the lack of a broader comparison, which makes it difficult to fully assess the method's contributions.", "questions": "Please refer to `Strengths And Weaknesses`.", "limitations": "The paper lacks sufficient experiments to demonstrate the benefits of the proposed method.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "### Strengths\n\n* **Clarity and Presentation:** The paper is well-written, clearly structured, and easy to follow. The figures are of high quality and effectively illustrate the core concepts and results.\n* **Novelty and Motivation:** The proposed method is novel and interesting. It is well-motivated with a clear physical explanation, which adds to the robustness of the approach.\n* **Strong Results:** The experimental results are strong and effectively demonstrate the potential of the proposed technique.\n\n### Weaknesses and Suggestions for Improvement\n\n1.  **Justification for Feature Conflict:** You claim that a conflict between anisotropic RGB and isotropic features leads to degraded results. This is a critical point that requires more substantial evidence. Please provide additional ablation studies or experiments to directly support this claim. Many existing works have demonstrated that jointly learning and optimizing different high-level features can be mutually beneficial for low-level recontructed result.\n2.  **Analysis of Performance Drop:** Similar to the point above, the results in Table 4 show that \"FHGS (w/o L_cf)\" performs worse than 2DGS. This is counter-intuitive, as one would expect the addition of feature rendering to maintain or improve performance. Could you provide a detailed explanation for this performance drop? It would be helpful to contrast this with other methods like Feature3DGS, where incorporating features does not seem to cause such a degradation.\n3.  **Expanded Comparison:** The evaluation would be more comprehensive if you compared your method against a wider range of recent, state-of-the-art approaches. Including more contemporary methods would provide a clearer context for your results and better highlight the specific advantages of your work.", "was_revised": true, "final_justification": "I have read the author rebuttal, and I will maintain my score.", "rebuttal": "We sincerely thank the reviewer for the insightful comments and constructive feedback. Your questions have helped us identify areas where our presentation could be clearer, and we greatly appreciate the opportunity to address these important points. \n\nOur method aims to differentiably drive 2D features into 3D space more effectively, outperforming native Gaussian Splatting (GS) in terms of speed, thereby preparing for subsequent tasks.\n\n\n**Response to Q1**: We sincerely apologize for any confusion in our presentation. We would like to clarify that we do not claim that anisotropic RGB and isotropic Feature-Fusion lead to degraded results. Actually, our work demonstrates that joint optimization of both achieves superior performance.\n\n*Our Core Argument*: We believe that features should not mechanically adopt RGB's differentiable, anisotropic rendering logic, such as LangSplat and Feature3DGS. Instead, we propose redesigning the approach based on two key characteristics of features: (1) Multi-view semantic consistency from large vision models (SAM); (2) The isotropic nature of semantic features.\n\n*Our Joint Optimization Strategy*: As shown in the overall loss in main paper: $L = L_{rgb}+ \\lambda_1 L_{gt}+\\lambda_2 L_{cf}$, where $\\lambda_1 = \\lambda_2 = 0.3$. The $L_{gt}$ and $L_{cf}$ serve as auxiliary losses alongside the primary RGB loss. This design enables Gaussian ellipsoids to maintain both multi-view semantic consistency and RGB's lighting-dependent rendering capabilities.\n\n*Our Design Rationale*:\n\n(1) Non-differentiable Features: We freeze SAM encoder weights because SAM features already possess multi-view semantic consistency, which may lack the interpretability of color gradients;\n\n(2) Isotropic Features: Using spherical harmonics for 32-dimensional features would likely cause memory explosion. Instead, we maintain isotropy and use sigmoid functions to handle SAM's potentially unstable cosine similarity.\n\n(3) Our method both aligns with the feature usage logic of SAM and emulates the positive and negative charge by sigmoid function classification in electric fields. \n\nWe hope this clarifies that our method successfully combines RGB and high-dimensional feature optimization, preserving 2D image features while extending them to 3D space for downstream tasks.\n\n**Response to Q2**: We appreciate the reviewer's observation about this interesting phenomenon in Table 4, which indeed reveals important characteristics of our physics-inspired approach. And this question also led us to a more in-depth reflection on our FHGS. \n\n*Understanding \"FHGS (w/o $L_{cf}$)\"*: This variant uses physics-based driving to project image features to homogeneous coordinates and compute electric field interactions with Gaussian ellipsoids (treated as charges), but WITHOUT inter-Gaussian charge interactions.\n\n*Possible Reasons for Performance Drop*: We conduct additional experiences and discover that SAM features' multi-view geometric consistency ranges from 0.65-0.85, which may lead to some issues:\n\n(1) Valuable Gaussian primitives being incorrectly deleted due to multi-view instability; \n\n(2) Surface contamination from erroneous duplication; \n\n(3) Over-deletion of Gaussian primitives potentially causing the PSNR drop.\n\n*How our Complete FHGS Addresses This*: The full FHGS computes similarity between Gaussian primitives($L_{cf}$), which we believe:\n\n(1) Increases gradients for contaminating Gaussian primitives, helping Adam optimizer re-evaluate descent; \n\n(2) May protect against erroneous deletion caused by SAM feature instability; \n\n(3) Enables rapid repositioning of misplaced Gaussian primitives.\n\nThis phenomenon also appears to manifest in outdoor scenes (Tab. 2), where even complete FHGS removes dark shadows, dynamic objects, and ambiguous features, resulting in PSNR reduction but potentially cleaner, more focused representations. We acknowledge this is an area that warrants further investigation.\n\n**Response to Q3**: \nWe greatly appreciate the reviewer's valuable suggestion for broader comparisons. After careful consideration, we would like to provide additional context for our experimental choices.\n\n*Current Feature Methods*:\n\nMost contemporary methods, including LangSplat and Semantic Gaussians, primarily focus on downstream applications of features (e.g., segmentation, multi-modal tasks) rather than the fundamental challenge of feature fusion itself. The existing feature fusion approaches largely follow Feature3DGS's paradigm - using MLPs for dimension reduction during training and adapting RGB rendering logic for feature fusion.\n\n*Rationale for Our Baseline Selection*:\n\nGiven this landscape, Feature3DGS represents the most relevant baseline method for our work as it directly addresses the feature fusion problem we tackle. This is why we chose it as our primary comparison in the main paper from quantitative and qualitative perspective. In contrast, methods like LangSplat and Semantic Gaussians focus more on multi-modal semantic fusion approaches, which represents a different research direction.\n\n*Future Directions*:\n\nWe are grateful for the reviewer's insight and comments, which has inspired potential future work. We could:\n\n(1) Design additional experiments specifically targeting multi-modal semantic features;\n\n(2) Integrate our FHGS results as input to LangSplat, Semantic Gaussians, and other downstream applications to potentially enhance their performance.\n\nHowever, we respectfully note that downstream applications and multi-modal tasks represent complex research areas that warrant dedicated investigation beyond the scope of our current work, which focuses on establishing a robust foundation for feature fusion in 3D Gaussian Splatting.\n\nWe hope this clarifies our experimental design choices and thank the reviewer for helping us consider these broader connections. We would be happy to include this discussion in the revised manuscript to better position our work within the field."}, {"review_id": "mwdSsBzbQQ", "reviewer": "Reviewer_qRwx", "summary": "This paper introduces FHGS, a framework designed to integrate high-level semantic features into 3DGS representation effectively. This work's primary motivation is to solve a fundamental conflict in existing 3DGS methods which is the contradiction between the anisotropic nature of color representation and the isotropic requirement for consistent semantic features. This could leads to a inconsistent semantics in 3D space. This paper addresses this by proposing a non-differentiable feature driving (NDFD), where FHGS directly compares the semantics of each primitive with ground-truth features from 2D image. The loss is designed to optimize not the feature but other gaussian geometric properties (position, rotation, scale, opacity, etc.) with the weight function introduced in the alpha blending process. Additional physics-inspired dual-drive optimization is proposed to ensure the arrangement of primitives and supresses the noise.", "questions": "**Q1**. Could you provide a qualitative or quantitative example of a downstream task where FHGS's high-quality features would provide a distinct advantage over prior work?\n\n**Q2**. In the process of adaptive density control (ADC) of gaussians, how is the feature handled?\n\n**Q3**. The method's performance appears to depend on several key hyperparameters (k and $\\lambda$) that were set empirically. Could you please provide some analysis or discussion on how sensitive the final results are to the specific values chosen for these parameters?\n\nAmong these points, my evaluation hinges most critically on the response to Q1. While clarifications on the feature handling during adaptive control (Q2) and a discussion on hyperparameter sensitivity (Q3) are certainly important for improving the paper's completeness and quality, demonstrating the practical utility of your high-quality features is essential for substantiating the work's significance.\n\nA compelling rebuttal that provides a clear example—either qualitative or quantitative—of FHGS's advantages in a downstream task would resolve my main reservation. In such a case, I would be strongly inclined to raise my rating to Accept or higher.", "limitations": "yes.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strength\n\n**S1**. Quality in writing: The paper is well-written and clearly structured. The authors do an excellent job of motivating the problem, presenting their methodology in a logical and easy-to-follow manner, and connecting their results back to their initial claims.  \n\n**S2**. Clear problem formulation and coherent solution: The paper's central premise is built on a sharp and insightful analysis of a fundamental conflict in 3D Gaussian Splatting; the contradiction between the anisotropic nature of color representation and the isotropic requirements of semantic features. This problem formulation is a contribution in itself, as it clearly articulates a core challenge for feature field learning in explicit representations. The proposed solution, which decouples the optimization of semantic features from their values by 'freezing' them and instead optimizing the geometric properties of the Gaussian primitives, is both novel and highly coherent. \n\n---\n\n## Weaknesses\n\n**W1.** Lack of demonstrated utility in downstream applications: The paper convincingly demonstrates that FHGS produces higher-quality feature fields with less noise. However, it stops short of showing why this is important in a practical sense. The question \"Why is it important to have *homogenized features*?\" remains partially unanswered. One way to address this is to show the utility of these feature fields for downstream tasks (e.g., semantic segmentation, text-prompt segmentation, editing, etc.).", "was_revised": true, "final_justification": "The author has addressed my primary weakness regarding the utility of the proposed method in downstream applications, demonstrating its potential for future research. The work is well-motivated, technically sound, and now has a clearer path for impact.\nBased on these points, I am raising my final score from 4 to 5.", "rebuttal": "We sincerely thank the reviewer for the thoughtful comments and constructive feedback. Your comments have helped us identify important aspects that deserve further clarification. \n\n**Response to Weakness**: We appreciate the reviewer's observation about demonstrating practical utility. The primary contribution of our work is to provide a more principled method for lifting 2D features to 3D space through differentiable optimization, while maintaining efficiency comparable to native 3D GS and 2D GS. We view this as foundational work that enables a wide range of future applications. We acknowledge that more downstream task demonstrations would strengthen our contribution and will consider this for the revised manuscript.\n\n**Response to Q1**: \nWe greatly appreciate this insightful question. The reviewer's suggestions are extremely crucial. Although our work is defined as fondimental research, it is indeed necessary to conduct further quantification through subsequent applications.\n\nBased on the reviewers' suggestions, we conducted the following experiments to compare 3D segmentation performance. We performed segmentation experiments using the LangSplat dataset, with segmentation results of other methods sourced from the paper **Semantic Gaussian**. We follow LangSplat to report localization accuracy (\\%) on 4 scenes. \nOur specific methodology is as follows:\n\n(1). Generate Semantic heatmaps using CLIP\n\n(2). Map heatmap pixels as rays into 3D space as 3D prompt\n\n(3). Compute Gaussian Splatting (GS) primitives intersecting with these rays\n\n(4). Identify neighboring GS primitives via K-means clustering\n\n(5). Determine segmentation clusters by calculating cosine similarity\n\nAs shown in the table, our results outperform state-of-the-art methods. \n\nNevertheless, our method effectively guides feature propagation in 3D space, making the superior 3D segmentation performance an expected outcome.\n\nOur method outperforms others in fundamental comparative experiments and possesses Zero-Shot capability. Analysis indicates that projecting the frozen SAM features into the correct 3D space significantly enhances segmentation performance. More importantly, our approach significantly outperforms other methods in terms of both training time and GPU memory consumption. \n\nTABLE \n| Method | ramen | figurines  | teatime | waldo kitchen |\n|:----------|:-----:|:-----:|:-----:|:-----:|\n| Semantic GS(CLIP)       | 76.8   | 83.1  |89.8 | 90.9 |\n| LangSplat       | 73.2   | 80.4  | 88.1 | **95.5** |\n| Feature3DGS       | 75.3   | 72.1  | 80.1 | 81.1 |\n| *ours(FHGS+SAM+CLIP)*       | **85.3**   | **89.5**  | **90.4** | 92.4 |\n\n*Future Directions*:\nUpon further reflection, we think the most promising direction is to develop 3D SAM Decoder architectures that can map from 2D SAM features to enable real-time 3D segmentation. Combined with FHGS's efficient feature lifting, this has potential applications in real-time scene understanding for robotics. We consider this a valuable avenue for future work.\n\n*Other Experimental Findings*:\nFor 2D segmentation tasks using rendered feature maps, our results are comparable to Feature3DGS. However, for 3D segmentation, FHGS shows significant advantages because we preserve the original feature data, making results more controllable. \n\n**Response to Q2**: \nWe thank the reviewer for highlighting this important implementation detail. During adaptive density control (ADC), features follow the same duplication and deletion logic as RGB values in native GS - they are copied or removed along with their parent Gaussian primitives. The key difference is that feature values are non-differentiable in our framework, meaning they remain unchanged during optimization. This ensures feature consistency while allowing spatial reorganization through ADC.\n\n**Response to Q3**: \nWe greatly appreciate this crucial question, which helps clarify these two hyperparameters and has inspired us to understand some underlying logic in SAM features.\n\nThe sigmoid function $\\sigma_i = \\frac{1}{1 + e^{k(\\varphi - \\lambda)}}$ serves in our physics-driven approach by classifying Gaussian primitives into positive and negative charges based on field-driven logic, used for similarity loss to drive features to their correct 3D spatial positions. \n\nThe hyperparameters $k$ and $\\lambda$ control the sigmoid function's slope and the position of axis of symmetry. Larger $k$ values create steeper slopes, providing greater discrimination of cosine similarities between Gaussian primitives.\n\n*Our Parameter Settings*:\nIn practice, we use $k=20$ and $\\lambda=0.7$. This manifests as:\n\n(1) At $x=0.6$, $y=2$, the function begins gradual decrease;\n\n(2) At $x=0.65$, $y=1.8$, the function begins rapid decrease;\n\n(3) At $x=0.8$, $y=0.15$, the rapid decrease ends;\n\n(4) At $x=0.85$, $y=0$.\n\n*How We Determined These Hyperparameters*:\nWe analyzed the cosine similarities of SAM features for the same SfM points across different views and found that 95\\% of points fall between 0.65-0.85. Based on this distribution, we designed the sigmoid function with appropriate slope.\n\n*Interpreting the 0.65-0.85 Similarity Range*:\nThis relates to SAM's training process. SAM first uses MAE for pre-training, utilizing its Image Encoder as the foundation. Through extensive data training, this Image Encoder has learned abundant general, separable, and multi-view consistent features. SAM then fine-tuned MAE's results for segmentation tasks, making boundaries clearer and more suitable for segmentation.\n\nWhile SAM's encoder features were not specifically designed for multi-view matching, their *moderate-to-high cross-view similarity* perfectly reflects the requirements: identical objects/scene parts across different views produce highly similar features (around 0.8), indicating robust high-level semantic and shape invariance for matching. \n\nSimultaneously, similarities not reaching extreme values >0.9 indicate features retain certain view-specific differences, helping maintain discriminability and avoiding confusion between different locations.\n\nThis effectively demonstrates that using SAM features directly for feature matching is feasible."}, {"review_id": "FSyPFk3GaW", "reviewer": "Reviewer_Xnyt", "summary": "The paper proposes FHGS (Feature-Homogenized Gaussian Splatting), a novel 3D feature fusion framework that addresses the inherent contradiction between the anisotropic color representation of Gaussian primitives and the isotropic requirements of semantic features in 3D Gaussian Splatting (3DGS). The key contributions include: 1）A universal feature fusion architecture for embedding 2D semantic features (e.g., from SAM, CLIP) into 3D structures. 2）A non-differentiable feature fusion mechanism to ensure viewpoint-independent isotropic feature distributions. 3）A physics-inspired dual-driven optimization strategy (external potential field + internal clustering) for global semantic alignment and local consistency.", "questions": "1. Comparisons are limited to 2DGS and Feature3DGS (before 2024). Recent works like LangSplat or Semantic Gaussians should be included to contextualize advancements.\n2. The paper focuses on feature fusion but does not evaluate downstream applications (e.g., segmentation, object detection) to showcase practical utility.\n3. The sensitivity to hyperparameters (λ1, λ2, k) is mentioned but not rigorously tested. A sensitivity analysis or adaptive tuning strategy would improve robustness.", "limitations": "Yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The introduction of non-differentiable feature fusion is a clever solution to reconcile the anisotropic rendering of 3DGS with isotropic semantic feature requirements. This is a significant departure from prior differentiable approaches.\n2. The physics-inspired dual-driven mechanism (electric field analogy) is creative and well-motivated, offering a principled way to enforce multi-view consistency.\n\nWeaknesses:\nThe paper presents a novel and technically sound approach to feature fusion in 3DGS, but its impact is limited by experiments.", "was_revised": true, "final_justification": "Thank you for your response. I have no further comments, but I recommend incorporating these discussions into the main manuscript to improve clarity and completeness.", "rebuttal": "We sincerely thank the reviewer for the constructive feedback and valuable suggestions. Your comments have helped us identify important areas for improvement.\n\n**Response to Weakness**: \nWe appreciate the reviewer's recognition of our novel and technically sound approach. We acknowledge that more extensive experiments would strengthen our contribution and will consider expanding our experimental evaluation in the revised manuscript, The primary contribution of our work is to provide a more principled method for lifting 2D features to 3D space through differentiable optimization, while maintaining efficiency comparable to native 3D GS and 2D GS.\n\n**Response to Q1**: We greatly appreciate the reviewer's valuable suggestion for broader comparisons. After careful consideration, we would like to provide additional context for our experimental choices.\n\n*Current Feature Methods*:\n\nMost contemporary methods, including LangSplat and Semantic Gaussians, primarily focus on downstream applications of features (e.g., segmentation, multi-modal tasks) rather than the fundamental challenge of feature fusion itself. The existing feature fusion approaches largely follow Feature3DGS's paradigm - using MLPs for dimension reduction during training and adapting RGB rendering logic for feature fusion.\n\n*Rationale for Our Baseline Selection*:\n\nGiven this landscape, Feature3DGS represents the most relevant baseline method for our work as it directly addresses the feature fusion problem we tackle. This is why we chose it as our primary comparison in the main paper from quantitative and qualitative perspective. In contrast, methods like LangSplat and Semantic Gaussians focus more on multi-modal semantic fusion approaches, which represents a different research direction.\n\n*Future Directions*:\n\nWe are grateful for the reviewer's insight and comments, which has inspired potential future work. We could:\n\n(1) Design additional experiments specifically targeting multi-modal semantic features;\n\n(2) Integrate our FHGS results as input to LangSplat, Semantic Gaussians, and other downstream applications to potentially enhance their performance.\n\nHowever, we respectfully note that downstream applications and multi-modal tasks represent complex research areas that warrant dedicated investigation beyond the scope of our current work, which focuses on establishing a robust foundation for feature fusion in 3D Gaussian Splatting.\n\nWe hope this clarifies our experimental design choices and thank the reviewer for helping us consider these broader connections. We would be happy to include this discussion in the revised manuscript to better position our work within the field.\n\nMore detailed comparison experiment results are shown in *Response Q2*.\n\nBased on the reviewers' suggestions, we conducted the following experiments to compare 3D segmentation performance. We performed segmentation experiments using the LangSplat dataset, with segmentation results of other methods sourced from the paper **Semantic Gaussian**. We follow LangSplat to report localization accuracy (\\%) on 4 scenes. \nOur specific methodology is as follows:\n\n(1). Generate Semantic heatmaps using CLIP\n\n(2). Map heatmap pixels as rays into 3D space\n\n(3). Compute Gaussian Splatting (GS) primitives intersecting with these rays\n\n(4). Identify neighboring GS primitives via K-means clustering\n\n(5). Determine segmentation clusters by calculating cosine similarity\n\nAs shown in the table, our results outperform state-of-the-art methods. \nNevertheless, our method effectively guides feature propagation in 3D space, making the superior 3D segmentation performance an expected outcome.\n\nOur method outperforms others in fundamental comparative experiments and possesses Zero-Shot capability. Analysis indicates that projecting the frozen SAM features into the correct 3D space significantly enhances segmentation performance. More importantly, our approach significantly outperforms other methods in terms of both training time and GPU memory consumption. \n\nTABLE \n| Method | ramen | figurines  | teatime | waldo kitchen |\n|:----------|:-----:|:-----:|:-----:|:-----:|\n| Semantic GS(CLIP)       | 76.8   | 83.1  |89.8 | 90.9 |\n| LangSplat       | 73.2   | 80.4  | 88.1 | **95.5** |\n| Feature3DGS       | 75.3   | 72.1  | 80.1 | 81.1 |\n| *ours(FHGS+SAM+CLIP)*       | **85.3**   | **89.5**  | **90.4** | 92.4 |\n\n*Future Directions*:\nUpon further reflection, we think the most promising direction is to develop 3D SAM Decoder architectures that can map from 2D SAM features to enable real-time 3D segmentation. Combined with FHGS's efficient feature lifting, this has potential applications in real-time scene understanding for robotics. We consider this a valuable avenue for future work.\n\nWe believe downstream applications deserve dedicated investigation. Our current work provides the foundation for such applications by ensuring high-quality, consistent feature fields in 3D space. \n\n**Response to Q3**: \nWe appreciate the reviewer raising this important point about hyperparameter sensitivity. Our method uses several key hyperparameters that deserve thorough analysis.\n\n*Loss Weight Parameters* ($\\lambda_1$, $\\lambda_2$):\nWe set $\\lambda_1 = \\lambda_2 = 0.3$ to balance the primary RGB loss with auxiliary feature losses. These weights were determined through preliminary experiments to ensure feature quality without compromising RGB rendering.\n\n*Sigmoid Function Parameters* ($k$, $\\lambda$):\nThe sigmoid function $\\sigma_i = \\frac{1}{1 + e^{k(\\varphi - \\lambda)}}$ serves in our physics-driven approach by classifying Gaussian primitives into positive and negative charges based on field-driven logic, used for similarity loss to drive features to their correct 3D spatial positions. The hyperparameters $k$ and $\\lambda$ control the sigmoid function's slope and the position of axis of symmetry. Larger $k$ values create steeper slopes, providing greater discrimination of cosine similarities between Gaussian primitives.\n\n*Our Parameter Settings*:\nIn practice, we use $k=20$ and $\\lambda=0.7$. This manifests as:\n\n(1) At $x=0.6$, $y=2$, the function begins gradual decrease;\n\n(2) At $x=0.65$, $y=1.8$, the function begins rapid decrease;\n\n(3) At $x=0.8$, $y=0.15$, the rapid decrease ends;\n\n(4) At $x=0.85$, $y=0$.\n\n*How We Determined These Hyperparameters*:\nWe analyzed the cosine similarities of SAM features for the same SfM points across different views and found that 95\\% of points fall between 0.65-0.85. Based on this distribution, we designed the sigmoid function with appropriate slope.\n\n*Interpreting the 0.65-0.85 Similarity Range*:\nThis relates to SAM's training process. SAM first uses MAE for pre-training, utilizing its Image Encoder as the foundation. Through extensive data training, this Image Encoder has learned abundant general, separable, and multi-view consistent features. SAM then fine-tuned MAE's results for segmentation tasks, making boundaries clearer and more suitable for segmentation.\n\nWhile SAM's encoder features were not specifically designed for multi-view matching, their *moderate-to-high cross-view similarity* perfectly reflects the requirements: identical objects/scene parts across different views produce highly similar features (around 0.8), indicating robust high-level semantic and shape invariance for matching. \n\nSimultaneously, similarities not reaching extreme values >0.9 indicate features retain certain view-specific differences, helping maintain discriminability and avoiding confusion between different locations.\n\nThis effectively demonstrates that using SAM features directly for feature matching is feasible.\nWe thank the reviewer for this constructive question, which has helped us better articulate our parameter selection rationale."}]

Author rebuttal:

Meta-review: This paper presents a well-motivated and highly novel contribution to the field of 3D Gaussian Splatting (3DGS). The work is founded on a critical and insightful observation of a fundamental, yet previously underexplored, limitation in existing 3DGS methods: the inherent conflict between the anisotropic nature of color splatting and the isotropic requirement for consistent semantic feature representation in 3D space.

The core strength of this paper lies in its coherent and elegant solution to this problem. The proposed non-differentiable feature driving (NDFD) mechanism is a conceptually innovative approach. Instead of directly optimizing semantic feature values—which would lead to the same inconsistencies—the authors propose to "freeze" the features and instead optimize the underlying geometric properties of the Gaussians (position, rotation, scale, opacity) to align with ground-truth 2D semantic features. This decoupling is a significant shift in perspective that effectively addresses the identified contradiction.

The supplementary physics-inspired dual-drive optimization further strengthens the framework by ensuring a physically plausible arrangement of primitives and suppressing noise, leading to a more robust and clean semantic representation.

The paper is exceptionally well-written, clearly structured, and does an excellent job of motivating the problem before logically presenting its methodological solution. The central problem formulation is itself a valuable contribution to the community, providing a clear theoretical foundation for future work in semantic 3DGS.

While the experimental validation and comparisons to baselines would be needed for a full assessment, the methodological novelty, soundness, and clarity of the proposed approach are themselves strong arguments for acceptance. This work provides a new and necessary direction for integrating high-level semantics into explicit radiance fields and is likely to influence subsequent research. The strengths significantly outweigh any potential weaknesses, making this paper a clear candidate for acceptance.
Decision: Accept (poster)