GENIE: Gaussian Encoding for Neural Radiance Fields Interactive Editing

Mikołaj Zieliński1, Krzysztof Byrski2, Tomasz Szczepanik2, Przemysław Spurek2, 3
1Poznan University of Technology, 2Jagiellonian University, 3IDEAS Research Institute
Paper arXiv Code Data Demo Data
Teaser Image

GENIE allows for editing of NeRFs by moving jointly trained Gaussians.

Abstract

Neural Radiance Fields (NeRF) and Gaussian Splatting (GS) have recently transformed 3D scene representation and rendering. NeRF achieves high-fidelity novel view synthesis by learning volumetric representations through neural networks, but its implicit encoding makes editing and physical interaction challenging. In contrast, GS represents scenes as explicit collections of Gaussian primitives, enabling real-time rendering, faster training, and more intuitive manipulation. This explicit structure has made GS particularly well-suited for interactive editing and integration with physics-based simulation. In this paper, we introduce GENIE (Gaussian Encoding for Neural Radiance Fields Interactive Editing), a hybrid model that combines the photorealistic rendering quality of NeRF with the editable and structured representation of GS. Instead of using spherical harmonics for appearance modeling, we assign each Gaussian a trainable feature embedding. These embeddings are used to condition a NeRF network based on the k nearest Gaussians to each query point. To make this conditioning efficient, we introduce RT-GPS (Ray-Traced Gaussian Proximity Search), a fast nearest Gaussian search based on a modified ray-tracing pipeline. We also integrate a multi-resolution hash grid to initialize and update Gaussian features. Together, these components enable real-time, locality-aware editing: as Gaussian primitives are repositioned or modified, their interpolated influence is immediately reflected in the rendered output. By combining the strengths of implicit and explicit representations, GENIE supports intuitive scene manipulation, dynamic interaction, and compatibility with physical simulation, bridging the gap between geometry-based editing and neural rendering.

Editing

GENIE makes it super easy to edit 3D scenes - just move or tweak the Gaussians directly! In this demo, the Gaussians are visually exaggerated to show how they follow the motion of the dozer and animate the scoop, all driven by simple mesh deformation.

Manual Edits

GENIE lets you manually adjust parts of a scene by moving Gaussians directly. Here, we tweaked the hotdogs on a plate to make it fly just by dragging points around.

Physical Simulations

With GENIE, you can apply physical simulations like soft body dynamics or cloth drops. This clip shows a Lego dozer being dropped, with the deformation driven entirely by a mesh-based simulation.

Real World Edits

This section shows two examples of real-world edits using GENIE. In one demo, a plant pot falls and bounces off a tilted table as part of a physics simulation. In the other, a soft plasticine Lego dozer is squashed by applying a force.

Interactive Demo

This demo lets you play with a pre-rendered animation of a fox from InstantNGP. Use the slider below to smoothly adjust the fox’s head position.

Loading...

NeuralEditor

In our paper, we compare GENIE with NeuralEditor. Here, you can see the NeRF-Synthetic objects edited in the same way.

Citation

If you find this work useful, please consider citing it: 

@misc{zielinski2025genie,
      title = {GENIE: Gaussian Encoding for Neural Radiance Fields Interactive Editing},
      author = {Miko\l{}aj Zieli\'{n}ski and Krzysztof Byrski and Tomasz Szczepanik and Przemys\l{}aw Spurek},
      year = {2025},
      eprint = {2508.02831},
      archivePrefix = {arXiv},
      url = {https://arxiv.org/abs/2508.02831}
}