GENIE: Gaussian Encoding for Neural Radiance Fields Interactive Editing
Authors: name: Mikołaj Zieliński; name: Krzysztof Byrski; name: Tomasz Szczepanik; name: Przemysław Spurek
Published: 2025-08-04 18:59:23

Abstract: Neural Radiance Fields (NeRF) and Gaussian Splatting (GS) have recently
transformed 3D scene representation and rendering. NeRF achieves high-fidelity
novel view synthesis by learning volumetric representations through neural
networks, but its implicit encoding makes editing and physical interaction
challenging. In contrast, GS represents scenes as explicit collections of
Gaussian primitives, enabling real-time rendering, faster training, and more
intuitive manipulation. This explicit structure has made GS particularly
well-suited for interactive editing and integration with physics-based
simulation. In this paper, we introduce GENIE (Gaussian Encoding for Neural
Radiance Fields Interactive Editing), a hybrid model that combines the
photorealistic rendering quality of NeRF with the editable and structured
representation of GS. Instead of using spherical harmonics for appearance
modeling, we assign each Gaussian a trainable feature embedding. These
embeddings are used to condition a NeRF network based on the k nearest
Gaussians to each query point. To make this conditioning efficient, we
introduce Ray-Traced Gaussian Proximity Search (RT-GPS), a fast nearest
Gaussian search based on a modified ray-tracing pipeline. We also integrate a
multi-resolution hash grid to initialize and update Gaussian features.
Together, these components enable real-time, locality-aware editing: as
Gaussian primitives are repositioned or modified, their interpolated influence
is immediately reflected in the rendered output. By combining the strengths of
implicit and explicit representations, GENIE supports intuitive scene
manipulation, dynamic interaction, and compatibility with physical simulation,
bridging the gap between geometry-based editing and neural rendering. The code
can be found under (https://github.com/MikolajZielinski/genie)