Neural Atlas Graphs for Dynamic Scene Decomposition and Editing
Authors: Jan Philipp Schneider
Pratik Singh Bisht
Ilya Chugunov
Andreas Kolb
Michael Moeller
Felix Heide
Venue: NeurIPS 2025 spotlight ()
Keywords: Scene Decomposition
Inverse Rendering
Neural Implicit Representations
Scene Reconstruction
Video Editing
Video Matting
Primary area: deep_learning
TL;DR: Neural Atlas Graphs are a hybrid representation enabling high-resolution texture and positional editing of dynamic scenes, achieving SOTA results on automotive datasets (Waymo) while demonstrating strong generalization on outdoor scenes (DAVIS).

Abstract: Learning editable high-resolution scene representations for dynamic scenes is an open problem with applications across the domains from autonomous driving to creative editing - the most successful approaches today make a trade-off between editability and supporting scene complexity: neural atlases represent dynamic scenes as two deforming image layers, foreground and background, which are editable in 2D, but break down when multiple objects occlude and interact. In contrast, scene graph models make use of annotated data such as masks and bounding boxes from autonomous-driving datasets to capture complex 3D spatial relationships, but their implicit volumetric node representations are challenging to edit view-consistently. We propose Neural Atlas Graphs (NAGs), a hybrid high-resolution scene representation, where every graph node is a view-dependent neural atlas, facilitating both 2D appearance editing and 3D ordering and positioning of scene elements. Fit at test-time, NAGs achieve state-of-the-art quantitative results on the Waymo Open Dataset - by 5 dB PSNR increase compared to existing methods - and make environmental editing possible in high resolution and visual quality - creating counterfactual driving scenarios with new backgrounds and edited vehicle appearance. We find that the method also generalizes beyond driving scenes and compares favorably - by more than 7 dB in PSNR - to recent matting and video editing baselines on the DAVIS video dataset with a diverse set of human and animal-centric scenes. Project Page: https://princeton-computational-imaging.github.io/nag/

Neural Atlas Graphs for Dynamic Scene Decomposition and Editing
Jan Philipp Schneider 1.2
Pratik Singh Bisht'
Ilya Chugunov?
Andreas Kolb'
Michael Moeller
Felix Heide
University of Siegen Prrinceton University
3 Lamarr Institute
"Tore Robotics
Abstract
Learning editable high-resolution scene representations for dynamic scenes is open problem with applications across the domains from autonomous driving to cre- ative editing- the most successful approaches today make trade-off between ed- itability and supporting scene complexity: neural atlases represent dynamic scenes two deforming image layers, foreground and background, which editable in 2D, but break down when multiple objects occlude and interact. In contrast, scene graph models make use of annotated data such masks and bounding boxes from autonomous-driving datasets to capture complex 3D spatial relationships, but their implicit volumetric node representations are challenging edit view-consistently. We propose Neural Atlas Graphs (NAGs), a hybrid high-resolution scene represen- tation, where every graph node is view-dependent neural atlas, facilitating both 2D appearance editing and 3D ordering and positioning of scene elements. Fit at test-time, NAGs achieve state-of-ther thetthe-rtt quantitative results on the Waymo Open Dataset by 5 dB PSNR increase compared to existing methods and make envi- ronmental editing possible in high resolution and visual quality - creating counter- factual driving scenarios with new backgrounds and edited vehicle appearance. We find that the method also generalizes beyond driving scenes and compares favorably bb more than 7 dB in PSNR to recent matting and video editing baselines the DAVIS video dataset with a diverse set human and unimal-centric scenes. Project computational itnptttiaaiiiaal-innnnnan imaging github. ctccubbio/nage io/nag/
Page:
1 Introduction
There has been growing demand in graphics and vision for high-fidelity static 3D 27 16 and dynamic 4D [35] 38) reconstruction models, and in particular for editable representations which decompose the scene into semantically meaningful components [43. For autonomous driving, where large collections of labeled video data required train driving behavior [13,, editable scene representations offer a direct approach to simulate counterfactual driving scenarios removing, re-timing, repositioning vehicles and pedestrians generate new trajectories, editing the visual elements of the scene to reflect new environmental conditions. This enables systems to expand limited collected real-world dataset into a richer and more diverse training set while preserving photo-realism and semantic consistency.
In this setting, neural scene graphs (29] have emerged as versatile hierarchical scene representation (339)455522) providing structured, object-centric models that enable repositioning and re-rendering elements with high visual quality. These methods model the scene set of connected nodes e.g.. vehicles, pedestrians, backgrounds represented as individual radiance fields [9] or collections of Gaussians (4) The nodes composited render views of the scene, optimized during test time fit an input driving sequence. Beyond visual data, these neural scene graph models can also ingest LiDAR and bounding box information, readily available in autonomous driving datasets [36] 47]
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

to better localize objects in 3D space over time. While nodes in the scene graph can be removed, rotated, or translated while preserving 3D view consistency, directly modifying their appearance is significantly more challenging. This requires altering the underlying Neural Radiance Fields (NeRF) or 3D Gaussian Splatring (3DGS) models, which requires method to propagate 2DD edits into 31) space [18] to preserve view-consistency.
Neural atlas representations [15] parallel line of work primarily focused on video editing, offer an alternative approach for this kind of appearance manipulation. These methods learn to map time- varying 3D environment to set of static owwrr-dimensione 2D "atlases", analogous to traditional UV unwrapping [12 of an object surface. During the process of fitting the scene, these models can disentangle a foreground object and its visual effects - e.g. the shadows it casts from its background. These atlases are then edited like a regular raster image, with changes to object textures propagating correctly through the video (0) Unlike neural scene graphs, however, these models do not represent scene elements explicitly in 3D space, and do not have an ordering between them 1 i.e., they cannot distinguish if one object is in front or behind another. When there are multiple overlapping objects in motion, neural atlas approaches resolve this by learning multiple non-overlapping alpha masks, with "ordering" achieved by the foreground mask cutting hole out the background mask [24]. While this does not pose problem for editing videos with single primary subject, for settings such as driving scenes, this makes it impossible to remove or reposition overlapping vehicles without introducing visual artifacts.
In this work, we introduce Neural Atlas Graphs (NAG) as a hybrid high-resolution representation without these limitations. Given input 3D bounding boxes and segmentation masks from autonomous driving stacks, NAG represents each scene element as a 2D plane with a 3D trajectory through space and time. Each of these planes acts as an independent neural atlas, capturing object motion, parallax, and lighting effects in a view-dependent neural field model. By explicitly modeling object depths and ordering, a NAG allows for flexible appearance editing at high resolution directly propagating changes from the 2D atlases to the reconstructed video without introducing distortions between occluded layers. Designed an inverse problem, the object trajectories and appearance fields are learned jointly, using ray-casting and efficient ray-plane intersections to accumulate colors and opacities along each ray. By fitting to recorded high-resolution images, accurate scene decomposition evolves naturally based on the varying motion patterns and provided masks. This enables our approach perform visually consistent removals, additions, rearrangements and texture editing of scene elements in complex, multi-object environments.
We validate the proposed method on automotive scenes 36) and confirm that the method outperforms recent object-specific 3DGS baselines in visual quality by almost 5 dB PSNR on overall scene quality, and up to 11.2 dB PSNR for dynamic objects. This confirms that the method able to learn accurate scene representations even under fast object motion, diverse reflections non-rigid motion patterns, while keeping positional and textual editability. For diverse outdoor scenes, with significantly less geometric prior available, covering various (non-)rigic actors, the method performs favorably to recent matting and neural atlas approaches with 7.3 dB PSNR margin, confirming the generality of our approach.
2 Related Work
Video Layer Decomposition Representing videos as a composition of individually deforming
layers is long-studied problem 40 with roots in seamless video editing 2 14). While more recent
works [46] achieve this via trained optical flow networks (37] and UNet-based masking [34], the core principles remain the same: estimate alpha masks and motion for dynamic elements to
separate them from a more static background for individual editing and re-composition. Atlas-style
methods - e.g., Unwrap Mosaics [32] and later Layered Neural Atlases [5]] learn 2D-to-2D
warps that map scene points onto an unwrapped canvas, similar to a UV map 12 Recent works
explore neural field [1] and neural radiance field [21] scene representations, in which compact networks are optimized at test time to map continuous coordinates in the input video sequence to view-consistent color. However, existing approaches assume videos that consist of primary object (and its visual effects, e.g., shadows) set against relatively static background, limiting their effectiveness complex, multi-object scenes. When multiple objects overlap, these methods rely on single-layer masks, resulting in visual holes where foreground elements cut into background

---

presents a method to generate realistic content to fill these occlusions, it reconstructs this content in the background canvas and does not model interactions between overlapping dynamic objects. In this work, we propose Neural Atlas Graphs with neural atlas representations that explicitly model multiple interacting layers, where point the scene can be mapped simultaneously to multiple time-consistent objects, enabling robust editing and re-composition even under complex occlusion scenarios.
objects 21I.. While Generative OmniMatte 20
3D Dynamic Scene Models Implicit representations such NeRFs [27 28 1 and explicit repre- sentations such as 3DGS [6
43 both enable high-fidelity photorealistic reconstructions of static scenes, and can be extended to fit dynamic scenes via learned deformation and flow fields [1] 35 43) However, in either case, editing the scene - and propagating those edits in a view-consistent manner -proves highly non-trivial [8 4 [ as these representations require changes to be carefully localized to avoid editing the wrong part of the seene, e.g., editing the subject and not their background. address this by factorizing the scene into per-object radiance fields, treated as graph nodes, enabling simple repositioning or removal of individual objects via edits to their corresponding nodes. This is made possible partially by the structured predictions of modern driving stacks [26] and annotations offered by autonomous driving datasets such as KITTI (10), nuScenes [3], and Waymo (36), which in addition to image data offer LiDAR point clouds, depth maps 3D bounding boxes, instance, and object or camera trajectories to instantiate graph nodes in 3D space. Recent hybrid methods combine scene graphs with 3DGS S to offer more efficient rendering times, but fine-grat fine-grained editing remains challenging, as changes to individual nodes must be propagated in a view-consistent manner, which remains an open problem. We propose hybrid representation that builds a 3D scene graph from structured bounding-bee box, mask, and trajectory data, but models each graph node as neural atlas (15), allowing both direct object appearance editing through manipulation of 2D canvas, and object repositioning in 3D space.
II
Neural Scene Graphs (29 45
3 Neural Atlas Graphs
Our proposed Neural Atlas Graph (NAG) illustrated in Fig. [ represents the scene graph of moving planes oriented in 3D space, with one plane per moving object - c.g., pedestrian, vehicle, bicyclist plus background plane. Each plane follows a learned rigid trajectory and carries surface-aligned optical-flow field together with view-dependent color-alpha maps capturing non- rigid motion, parallax, and illumination changes. Rendering is done via depth-ordered ray-casting and alpha compositing across the planes. The subsequent sections detail these components.
3.1 Representation
Our Neural Atlas Graph (NAG) representation takes as input an arbitrary video scene I E R X comprised of F 3-channel RGB images of size W x H, and stack of coarse masks M E {0, 1xx xefxxWXli XHXN corresponding to N-many foreground objects nodes in our graph. The remaining unmasked region is represented with an additional background node. To establish the initial position, rotation, and ordering of nodes within the scene graph we primarily rely on the orientation of supplied 3D bounding boxes. If 3D bounding boxes are not available, we fall back to homographies determined from monocular depth estimation for plane initialization [44]. Unfor- tunately, dynamic scene models such as Neural Atlas Graphs (NAG) inherit the same scene and camera motion ambiguities as found in visual Simultaneous Localization and Mapping (vSLAM) problems. Therefore, we also require an initial estimation of the camera extrinsics and intrinsies, similar to common NeRF (27] and 3DGS [16] approaches. We represent each node (N.) tuple Nr = (ci, 044, fr, gr, 8;) containing color e; : [o, 1]2 + R3, opacity a; [0, 1]2 + R. optical flow field fi [0, 1 x [0, 1] + RR, time-dependent position gi 1] p4xx and plane size
8; E IR?.
3.2
Image Formation
NAG are rendered with a forward ray-projection model. Each pixel intensity at a timestep E [0, 1] in the image I is composed by aggregating radiance at typically a handful of plane/ray intersections for rays with direction d TR3 and origin o e R3 (see supplementary material for details). Given a planar node at position gt, decomposable into position p and normal vector n, and the ray r(I) + ld.
3

---

(a)
Eackground
(b)
Neural Adlas
Sceue
Alpha Malte
Camera
Node 2
View Fields
288
Flow
Motion
Finall Color
Node
Rendering:
4
Neural Atlas Graphs A NAG represents dynamic scenes (a) as graph of moving 3D planes (one per object/background) Each plane undergoes rigid transformations and encodes view- dependent appearanceerransparenyy using neural fields F (b) along a learned trajectory J:- The planar optical flow fi models non-rigid motion and parallax, while learning the representation and rendering is done via opacity-weighted ray casting of Ci,,, Ai,t using position based z-buffering.
Figure 1:
the ray-plane intersection - (p o) n/(d (dd--v n) may yield an intersection point] Tweeld + ld. When projected a finite sized plane i, resulting in x E [o, 1]2 (Sec. (.3). and applied for each node in the NAG, the object color Ci,1 and opacity Ai, at time t can be determined via planar optical flow field fx:
Ci c fi(x,)), = = a; f((x,t)).
(1)
We coppa-composite 17
each plane intersection along the ray, yielding the final per-pixel color C'
N C= C=
(1-AA.)) f=0
1=0
of each ray. Given all object colors C.,t and opacities Ar, have been ordered by the distance to the camera in ascending order. If ray does not hit the corresponding plane falling off the edge of the finite extent s; of the plane- we set As,t = O.
3.3 Parametrization Neural Atlas Nodes
Next, we describe the parametrization of the model components inside nodes N (ci, at fi. It. 8:). We opt for modeling our NAG nodes in 3D space, where each planar node is assigned a 3D position and orientation with 6 DoF assuming rigid motion model.
Rigid Plane Pose [9.] The affine rigid transformation matrix (t) encodes the node's trajectory over time. We decompose this matrix into translation (t(t) and rotation (t) components, which learned independently. To ensure temporal coherence, use smooth Hermite splines
Tt(t) Ti, + Stt, P.).
Here, we learn an offset relative to an initial position T, e R Fx3, When 3D bounding boxes are available, Tú, corresponds to the box center at time t.
The function S [o. 1] R F denotes the piecewise cubic Hermite spline interpolation of P number of inro-iniiaaiieed, learnable control points P. E R The weight = 0.5 controls the contribution the spline. Adjusting the number of control points P determines the smoothness of the trajectory. For rotation (t),,, we use the orientation of the bounding box with an added offset" When bounding boxes are not available, we instead learn the rotation using the following offset model
R (t) R., q/e S(t. )))).
'Given non-paran non-parallel rays, e.g. d n F 0, and assuming not within the plane. "The offset is chosen to align the plane with the box's 's front, side, or diagonal face whichever yields the smallest inclination to the camera view. This allows planar representations even for turning objects, like cars, as long their rotation remains under 180

---

given an initial rotation R, E H'D we apply a rotation offset via the rotation cerioon-vetoo--eee rector-to-quuee to-quatern mapping q : [0, 2 + H. using zero- vero-initiaaizze learnable offsets PR Rxx3 In the absence of bounding boxes, both T, and RG are estimated by decomposing per-object image homographies [25) into relative 3D translations and rotations. These are then cumulatively applied to planar projection of a monocular depth estimate, yielding per- per-frame pose estimates.
Plane Extent [s:] To prevent merging of distinct objects exhibiting similar rigid motion and to improve efficiency, we constrain each plane to a finite extent. A point Tworld E IR in homogeneous world coordinates lies the finite plane of object i if the coordinates of its planar correspondence point I = (gx(t)))wwota Zworld + 0.5) [Sa Sys 0, OJT are within [0, 1] range, where denotes element-wise multiplication.
The plane scale s; = {sz, sy is pre-estimated from the largest mask within the stack M. plus a relative margin to incorporate object-associated effects, e.g., shadows, and mask inaccuracies.
Node Color [C,,]] and Opacity [A,,]] Given ray-plane intersection point [o, 1]2 and its spherical view-angle é E [0, 1]2 in the plane coordinate system, we model the color and opacity of each node as the combination of a base color field C₁= [0, 1]2 - R and opacity field A, [0. 1]
R. each augmented by two separate learned neural fields. The base appearance, estimated via forward projection onto masked regions, is parameterized on a pixel grid and extended to arbitrary
00, 1]
via bilinear interpolation. We use two types of neural fields: view-agnostic fields Fs, [0, 1] + R2 and F..a+ [0, 1]2 + E. and view-dependent fields Fia.. [0. 1] + R and Fi, : 1]
R.
the latter implicitly regularized via a coarse-to-fig scheme This combination balances editability requiring time-consistent atlas content and adaptivity to scene dynamics, lighting, and view direction, resulting in both temporal stability and high visual fidelity. Our representation is then
(x) (z) + ne Fr,(F) + no Fr,,,e(1,)) Ø),
(5)
(x(x) log
1) + na FF,a(z) + no (2,),(z,
(6)
(x(x)
which we use in (1) to represent C₁,t and Ai,t While using the base estimates C, A, allows to precondition the object learn, the subsequent MI.s Fi,e,Fio including positional encodings (28. refine projection errors and learn temporally consistent, editable representation. To capture view- or time-dependent appearance changes, we introduce an additional MLP Fi, which predicts color and opacity offsets based the planar point and its associated spherical view angle é at the ray-plane intersection. We weight the contributions of the networks using fixed scalars T/e = 7/ = Nl = 0.1, and enforce valid ranges by clamping color values and applying sigmoid to opacity, ensuring (z(z), (z) E [0, 1].
Flow Field [fi] We rely temporally changing flow field f. [0, 1] - 1 attached to each node to model small non-rigid deformations and depth-induced parallax. c.f. 5 To ensure smoothness, we use a spline-based flow model [6), which shifts the ray-plane intersection point before is passed to the subsequent networks. That is
M(x, () ny
The flow field Fir [0, 1|2 + P2 is implemented an MLP similar to the color and opacity networks. However, instead of predicting a single flow vector, it outputs a set of Pr control points, which are interpolated using cubic Hermite spline S(t,-) (cf. Sec. 3.3) to produce smooth, varying flow. We employ a coarse-to-fin to-fine fitting strategy by progressively masking the encoding dimensions (see supplementary material). Similar to color and alpha, the flow is modeled an offset and scaled by a fixed weight ny = 0.1.
3.4 Background Atlas
In addition to the dynamic foreground objects, we model the background using a dedicated node covering all non-masked regions. The background is represented as a fixed planar atlas with no
PHt denotes the set of unit quaternions.
We detail the coarse-to tor--t--fiin fine scheme, along with phase-based learning within the supplementary material.
5

---

rigid transformation or opacity variation. Its global pose 9B E R4x4 is held constant throughout the sequence and placed behind all object bounding boxes. Its orientation and size are chosen to ensure full visibility across the entire camera trajectory, including rotations and translations. The background has the same color and flow model as the foreground nodes (see ( (7)). but uses fixed opacity Ag and omits rigid motion to maintain consistent reference frame.
3.5 Optimization
We fit the model by sampling bani-batches of random pixels and timestamps from the input video as ground truth values y E I and render color predictions ý. We jointly optimize the transformation parameters and network parameters of all nodes with the loss
Catlas
|| e|li + 3 - -m
(8)
which combines photometric & term with mask loss that compares the predicted opacity à to the input mask m E M. The mask term encourages objects to remain opaque in masked regions while allowing flexibility in unmasked areas to represent shadows object-induced effects. We empirically set 3 = 0.005 to balance these objectives.
3.6
Atlas Texture Editing
To edit the appearance of an object (or the background), we require a user-provided RGBA texture with color è E [0, 1|W x x3 and alpha E [0, 1]] within the image space of a dedicated reference image Ire This texture gets ray-projected onto the planar object M₁. On the plane, numerically invert the learned flow model and bilinearly interpolating values, to store the user texture sample the texture at arbitrary ray-plane intersection points x. Further, we can blend the values at č(x)) (xx) with the original color model (5) via alpha matting,
é: [o, 1j2 + R and à [0, 1)2 Ih regular grid in atlas space. This allows
- - (1-áz)) cx(x) + á(x) E(x),
to our learned object representation. Using of instead of c within allows us to render the scene including the asovidrrvided texture.
Storing the texture representation within atlas space is critical: it ensures the edit is independent of the camera motion (given its definition on the node, which undergoes rigid motion) and enables view-consistent editing of subsequent frames as long as these are accurately modeled within our flow representation. Furthermore, by defining textures in image space and projecting them, we allow for better edit comparability since textures are defined independently of the learned flow, in contrast to [15]. This design also enables the seamless use of standard image editing tools and off-the-shelf image generation models.
4 Experiments
To evaluate the efficacy and versatility of the proposed method, we conduct three distinct experiments. First, we assess the method with quantitative and qualitative comparisons to Mfte-offthhe-rt driving scene reconstruction techniques
43 Next, we validate and compare the editing capabilities, such object removal, and texture editing, challenging autonomous driving scenes. Finally, we assess the generalization of the approach on diverse non-driving outdoor scenes.
4.1 Driving Scene Reconstruction
Setup For evaluating the quality of our method in autonomous driving scenarios, rely subset of the Waymo (36] open dataset. We select scenes with small ego motion but many objects, occlusions, and diverse and large object motions. In total, we evaluate on scene segments including up to 199 images each, sampled at 10Hz using the forward camera feed. We divide each data segment into sequences of 21 to 89 images, leading to 25 subsequences to be reconstructed.
We filtered the 3D bounding boxes provided for actually moving objects and used the sparsely provided instance segmentation masks in a semi-automatic process using SAM2 (33 to get reasonable
masks for every image. Notably, an exemplary ablation study in the supplementary material indicates that our approach maintains applicability even when using coarser masks only bounding boxes.

---

Representative ground truth images of these dataset sequences, including our masks and further training details can be found our supplementary material or code page: httpp:://gtiie com/ jp-schnei der/nag
Assessment We evaluate our method against the recent OmniRe 4 method, an object-centric 3DGS approach with explicit SMPL [23] human modeling and EmerNeRF [43] recent NeRF method including static and dynamic radiance fields 5 Tab. reports PSNR, SSIM [2] and LPIPS 49 on
the individual scenes. The method consistently outperforms the baselines, reaching a significant 5 dB PSNR mean increase over OmniRe and 7 dB over EmerNeRF. To assess these differences visually, we show comparisons in Fig. 2 but recommend viewing the full-resolution images and videos provided in the supplementary. Especially under rapid motion of objects, OmniRe [4] tends to produce artifacts. This can become apparent in missing or smoothed out objects edges, foots of eyclists, partially visible side-mirrors, missing distant objects or reflections which are more reliably modelled with our approach. EmerNeRF 43 tends to render objects over-smoothed, indicating under-parann parammtttatttinsioonns due to its scene representation in a static and dynamic radiance field. Our NAG node formulation able to precisely model fine nigh-resolution details, such as spinning wheels, pedestrian motion, reflections or objects in the distance, even if they are part of the background plane.
Table 1: Quantitative Evaluation on Dynamic Driving Sequences of the Waymo (36 Open Driving Dataset. Best results are in bold. ORe refers to OmniRe [4, and ERF to EmerNeRF 43
PSNR 1
SSIM f
LPIPS +
Seq.
Ours
ORe
ERF
Ours
ORe
ERF
Ours
ORe
ERF
s-975
40.21
37.35
34.83
0.976
0.968
0.937
0.058
0.080
0.143
s-203
43.15
36.93
36.07
0.978
0.966
0.936
0.070
0.094
0.205
s-125
43.32
38.74
35.20
0.980
0.970
0.933
0.057
0.079
0.182
s-141
42.55
36.14
34.83
0.978
0.964
0.924
0.057
0.087
0.178
s-952
41.89
39.67
35.32
0.976
0.977
0.938
0.058
0.050
0.120
s-324
40.85
32.58
33.63
0.977
0.953
0.926
0.038
0.071
0.124
S-344
41.84
36.67
35.24
0.983
0.973
0.946
0.031
0.043
0.084
Mean
41.85
36.78
34.93
0.978
0.967
0.934
0.051
0.070
0.142
We further evaluate the quality of the dynamic objects in isolation, grouped into a rigid "Vehicle" class and non-rigid "Human" class, which allows to differentiate between objects that may benefit differently from our underlying rigid-motion motion model. Tab. 2 validates significant improvements over the best baseline, with PSNR gains of 11.2 dB and 10.7 dB, and SSIM increases of 0.064 and 0.080 for the Vehicle and Human classes, respectively. These findings confirm that the quality of our method does not stem from better background rendering but indeed from the high accuracy of our model in representing even non-rigidly moving actors.
For detailed introduction on our baselines we refer supplementary material.
Table 2: Quantitative Evaluation of Human and Vehicle Rendering Waymo 36
Driving Sequences.
Vehicle PSNR 1
Vehicle SSIM f
Human PSNR 1
Human SSIM 1
Seq.
Ours
ORe
ERF
Ours
ORe
ERF
Ours
ORe
ERF
Ours
ORe
ERF
s-975
46.79
33.09
30.21
0.991
0.939
0.82
45.37
32.99
28.53
0.989
0.927
0.777
s-203
41.90
30.45
27.10
0.986
0.910
0.774
45.40
34.85
33.54
0.986
0.950
0.901
s-125
41.00
28.72
24.55
0.989
0.878
0.709
N/A
N/A
N/A
N/A
N/A
N/A
s-141
43.21
33.22
27.36
0.981
0.929
0.744
44.22
33.31
28.86
0.986
0.907
0.769
s-952
40.94
31.15
27.70
0.986
0.928
0.810
40.45
32.32
28.10
0.968
0.894
0.740
s-324
41.71
31.03
27.87
0.986
0.921
0.798
44.12
32.09
26.40
0.988
0.894
0.689
s-344
43,97
33.02
30.65
0.985
0.931
0.835
40.99
30.20
25.94
0.975
0.882
0.721
Mean
42.88
31.69
28.09
0.986
0.922
0.787
42.94
32.24
27.78
0.981
0.901
0.744
7

---

bil
Ground Truth
Ours
OmniRe
EmerNeRF 4
Figure 2:
Visual quality comparisons (sequences s-952, 344, 975) show that method achieves higher fidelity by producing way fewer artifacts on rapid motion (e.g., spinning wheels, edges), significantly reducing motion blur, and preserving finer details like reflections.
Scene Editing We showcase a scene decomposition task in Fig. 3 where all dynamic actors are extracted and separated from the background Comparing to OmniRe, significantly fewer artifacts around the object peripheries, adding correct shadows and keeping distant information. Given our graph approach, the graph and render these into our scene. These functions, along with an evaluation on temporal consistency, are demonstrated in supplementary material. Also, given our planar nodes, can take arbitrary textures in image space and project them onto one or multiple nodes to change their appearance along time, as demonstrated in Fig. 4 By combining object removal, and background texture editing, we can generate realistic counterfactual driving scenes. To demonstrate this capability, we execute realistic editing task within the Waymo s-203 sequence, shown Fig. Here, the car integrates naturally with the edited street, correctly incorporating painted speed signs and zebra crossing. Crucially, the shadow casts realistically along these edited areas, showcasing our method's 's ability to maintain natural occlusion behavior. Furthermore, the seamless removal of both the person the foreground and the car in the background, without recognizable artifacts, highlights the precision required for reliable testing within autonomous driving stacks.
method suffers from
further remove, shift, and add additional nodes to
Ours
OmniRe
Ground Truth
Ours
OmniRe 4
Figure 3: Scene Decomposition on Waymo (sequences 125 and 141). Visual comparison demon-
strates our method's capability for accurate object decomposition, preserving sharp boundaries and shadows for both rigid and non-rigid entities. Furthermore, our approach effectively maintains object information even under occlusion and adverse weather conditions.
For clearer illustration, please refer the supplementary videos.

---

Figure 4: Consistent Appearance Editing. We showcase the consistent application of a textures (left) onto dynamically moving objects, changing their appearance and model occlusions accurately.
25
Figure 5: Effective manipulation in s-203. Four timestamps from the s-203 scene shown, with ground truth (top) and our edits (bottom). The applied background texture is presented top-right, and the removed objects shown bottom-right. The natural blending of the car and its shadow with the edits demonstrates our realistic occlusion handling, showcasing our methods applicability in creating counterfactual driving scenarios.
4.2 Outdoor Video Scenes
We validate the generalization of our method on diverse outdoor scenes from the DAVIS dataset [30. which has been regularly used by matting methods [4] 21 and provides high-resolution images (up to 1920 x 1080). We evaluate on 15 specific sequences following 15 21.. To ensure comparability between our method and the baseline approaches, we match the evaluation setups of all methods, see Supplemental Material for details.
We report our results in Tab. 3 and find an average 7 dB PSNR improvement w.r.t the best baseline. than in the automotive scenes, due complex camera and object motion, the method also outperforms all baseline approaches for all sequences. Notably, when objects exhibit rigid-motion behavior (e.g., blackswan, car-shadow kite-surf), achieves its best results. Furthermore, even in scenes involving non-rigid motion (e.g., bear, hike, and elephant), where the challenges more pronounced, we still outperform the state-of-the- baselines, validating the generalization to outdoor scenes with various actors. We provide additional baseline information, visual examples, and scene decompositions, in our supplementary material.
While the overall performance varies
method
4.3 Limitations
Our method relies on the assumption that objects and background can be projected onto plane, imposing limits on camera and object rotation (less than 180 degrees). Correspondingly. Novel View Synthesis and accurate 3D representation is not within the scope of our current manuscript. Accurate initial plane initialization via bounding boxes or geometric priors is recommended, and suboptimal initialization can lead to early failure. Projection errors on non-planar object geometry may accumulate and hinder precise initial position determination.
Representing large camera motion that significantly changes the background is also challenging for the proposed plane assumption, and further studied the supplementary material. This due to the difficulty of capturing large flow vectors, particularly when sampling rays from only subset of time steps and coordinates as we do. However, we note that the use of view dependency compensates for these errors, although does come at the cost of reduced texture editability.

---

Table 3: Quantitative evaluation results the Davis Dataset 30 of diverse outdoor scenes. Our method consistently outperforms the Layered Neural Atlases (LNA) [1]] and OmnimatteRF (ORF) 21 baselines, achieving the best metric scores along all sequences. Also scenes with intricate non-rigid motion where alignment is learned are more demanding, our method still yields superior results, with particularly significant gains in sequences well-suited to our motion model, such as motorbike, kite-surf, and car-shadow.
PSNR f
SSIM f
LPIPS +
Sequence
Ours
ORF
LNA
Ours
ORF
LNA
Ours
ORF
LNA
bear
33.47
24.88
26.51
0.934
0.658
0.771
0.091
0.464
0.287
blackswan
36.36
26.67
29.26
0.938
0.739
0.815
0.097
0.458
0.318
boat
35.83
28.63
30.15
0.932
0.761
0.816
0.099
0.376
0.274
car-shadow
36.67
29.26
28.47
0.947
0.861
0.850
0.084
0.313
0.269
elephant
33.91
26.94
28.34
0.922
0.731
0.772
0.088
0.423
0.325
flamingo
34.96
25.74
27.10
0.928
0.753
0.783
0.106
0.483
0.349
hike
29.74
25.15
24.77
0.886
0.698
0.682
0.108
0.388
0.343
horsejump-higee
34.78
28.35
27.28
0.932
0.846
0.830
0.074
0.249
0.226
kite-surf
37.96
28.04
27.88
0.949
0.780
0.780
0.068
0.420
0.400
kite-walk
37.96
29.44
29.58
0.941
0.804
0.818
0.070
0.367
0.334
libby
38.89
29.62
29.35
0,949
0.819
0.828
0.095
0.399
0.342
lucia
30.90
26.03
26.63
0.869
0.690
0.742
0.178
0.407
0.329
motorbike
37.42
27.33
29.33
0.950
0.779
0.843
0.082
0.376
0.241
swing
35.70
26.14
27.88
0.926
0.722
0.808
0.119
0.404
0.289
tennis
35.65
27.43
28.81
0.928
0.806
0.862
0.120
0.328
0.209
Mean
35.35
27.31
28.09
0.929
0.763
0.800
0.098
0.390
0.302
Conclusion
In this work. introduce Neural Atlas Graphs (NAGs), editable hybrid scene representation for high-resolution learning and rendering of dynamic scenes. We find that the hybrid 2.5D representation of NAGs compares favorably in representing driving scenes. Specifically, achieves state-of-the-ar art results on the Waymo Open Dataset, with 5 dB PSNR improvement overall and 11.2 dB PSNR for rigid and 10.7 dB PSNR for non-rigid actors. We show that NAGs enable high- resolution, view niw-ccosisttent environmental editing, unlocking the creation of compelling counterfactual driving scenarios and showcasing potential not only in scene arrangement and scene decomposition but also in consistent appearance editing. We also confirm the generalization of NAGs beyond understanding and manipulation with evaluation the
validate that the method
driving scenes for comprehensive outdoor DAVIS video dataset, achieving 7.3 dB PSNR improvement over existing methods. In the future. the differentiable representation with the low-dimensional neural atlases may allow for task-driven editing, such as learning of counterfactuals specifically to challenge driving stack, which we exemplary demonstrated Fig.
Acknowledgments
We extend our sincere gratitude to Julian Ost, Mario Bijelic, Amogh Joshi, and William Koch from Princeton University for their insightful contributions through numerous discussions concerning the autonomous driving context, recent scene representation literature, and methodologies. This research was supported by the DFG Research Unit 5336 Learning to Sense (Project No. 459284860) and project funding for "Polymorphic Scene Representation for Enhanced Instant Scene Reconstruction" (Project No. 510825780, Ref. KO 2960/20-I) 1). Ilya Chugunov was supported by NSF GRFP (2039656). Felix Heide supported by an NSF CAREER Award (2047359), Packard Foundation Fellowship, a Sloan Research Fellowship, a Disney Research Award, a Sony Young Faculty Award, a Project X Innovation Award, Amazon Science Research Award, and Bosch Research Award.
10

---

References
Jonathan T Barron, Ben Mildenhall, Dor Verbin, Pratul P Srinivasan, and Peter Hedman. Zip-nerf: Anti- aliased bised neural radiance fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 19697-19705, 2023.
[2]
Christoph Bregler, Michele Covell, and Malcolm Slaney. Video rewrite: Driving visual speech with audio. In Proc. SIGGRAPH, pages 353-360, 1997.
[3] Holger Caesar, Varun Bankiti, Alex H. Lang. Sourabh Vora, Venice Erin Liong. Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom, nuscenes: A multimodal dataset autonomous driving. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pages 11621-1163 1631,, 2020.
[4] Ziyu Chen, Jiawei Yang, Jiahui Huang, Riccardo de Lutio, Janick Martinez Esturo, Boris Ivanovic, Or Litany, Zan Gojeic, Sanja Fidler, Marco Pavone, Li Song, and Yue Wang. Omnire: Ommi urban scene reconstruction. The Thirteenth International Conference on Learning Representations, 2025.
[5] Chong Cheng, Gaochao Song, Viyang Yao, Qinzheng Zhou, Gangjian Zhang, and Hao Wang. Graph- guided scene reconstruction from images with 3d gaussian splatting. In Proceedings of the International Conference on Learning Representations (ICLR), 2025.
[6] Ilya Chugunov, David Shustin, Ruyu Yan, Chenyang Lei, and Felix Heide. Neural Spline Fields for Burst Image Fusion and Layer Separation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 25763-25773, 2024.
[7] Carl De Boor, Klaus Höllig, and Malcolm Sabin. High Accuracy Geometric Hermite Interpolation. Computer Aided Geometric Design, 4(4):269-277 278, 1987. Publisher: Elsevier.
[8] Jiahua Dong and Yu- Xionio Wang. Vica-nerf: fiow-consistency-wwal 3d editing of neural radiance fields. Advances in Neural Information Processing Systems, 36:61466-61477, 2023.
[9] Tobias Fischer, Lorenzo Porzi, Samuel Rota Bulo, Mare Pollefeys, and Peter Kontschieder. Multi-level neural scene graphs for dynamic urban environments. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21 125-21135, 2024.
[10] Andreas Geiger, Philip Leaz, and Raquel Urtasun. Are we ready for autonomous driving? the kitti vision benchmark suite. In 2012 IEEE conference computer vision and pattern recognition, pages 3354-3361.
IEEE, 2012.
[11] Antoine Guédon and Vincent Lepetit. Sugar: Surface-aligned gaussian splatting for efficient 3d mesh reconstruction and high-quality mesh rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5354-5363, 2024.
[12] Paul S. Heckbert. Survey of texture mapping. IEEE Computer Graphics and Applications, 6(11):56-67, 1986.
[13] Jyh-Jing Hwang. Runsheng Xu, Hubert Lin, Wei-Chih Hung. Jingwei Ji, Kristy Choi, Di Huang, Tong He, Paul Covington, Benjamin Sapp. Yin Zhou, James Guo, Dragomir Anguelov, and Mingxing Tan. Emma: End-to to-end multimodal model for autonomous driving. arXiv preprint arXiv:2410.23262 2024.
[14] Nebojsa Jojie and Brendan J. Frey. Learning flexible sprites in video layers. In Proc. IEEE Conf. Computer Vision and Patiern Recognition, pages 199-206, 2001.
[15] Yoni Kasten, Dolev Ofri, Oliver Wang, and Tali Dekel. Layered neural atlases for consistent video editing. ACM Transactions on Graphics (TOG), 40(6) 122 2021.
[16] Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3D Gaussian Splatting for Real- Time Radiance Field Rendering. ACM Trans. Graph., 42(4):139 2023.
[17] Georgios Kopanas, Thomas Leinkühler, Gilles Rainer, Clément Jambon, and George Drettakis. Neural Point Catacaustics for Novel- View Synthesis of Reflections. ACM Trans. Graph., 41),), 2022, Publisher: Association for Computing Machinery.
[18] Zhengfei Kuang, Fujun Luan, Sai Bi, Zhixin Shu, Gordon Wetzstein, and Kalyan Sunkavalli. Palettenerf: Palette- based appearance editing of neural radiance fields. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20691-20700 2023.

---

[19] Abhijit Kundu. Kyle Genova, Xiaoqi Yin, Alireza Fathi, Caroline Pantofaru, Leonidas J Guibas. Andrea Tagliasacchi, Frank Dellaert, and Thomas Funkhouser. Panoptic neural fields: A semantic object-aware neural scene representation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12871-12881 2022.
[20] Mao-Chih Lee, Erika Lu, Sarah Rumbley, Michal Geyer, Jia-Bin Huang. Tali Dekel, and Forrester Cole. Generative omnimatte: Learning to decompose video into layers. arXiv:2411.16683 2025.
[21] Geng Lin, Chen Gao, Bia-Bin Huang. Changil Kim, Yipeng Wang, Matthias Zwicker, and Ayush Saraf. Omnimatterf: Robust omnimatte with 3d background modeling. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 33471-33480, 2023.
[22] Jeffrey Yunfan Liu, Yun Chen, Ze Yang. Jingkang Wang, Sivabalan Manivasagam, and Raquel Urtasun. Real-time neural rasterization for large scenes. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8416 6-1277,,, 2023.
[23] Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Mol,, and Michael J. Black. Smpl: A skinned mulli-pprsoo lincar model. In Seminal Graphics Popers: Pushing the Boundaries, Volume 2, pages 851-866. Association for Computing Machinery, New York, NY, USA, I edition, 2023.
[24] Erika Lu, Forrester Cole, Tali Dekel, Andrew Zisserman, William T. Freeman, and Michael Rubinstein. Omnimatte: Associating objects and their effects in video. In Proceedings of the IEEF/CVE Conference on Computer Vision and Pattern Recognition (CVPR), pages 4507-4515, 2021.
[25] Ezio Malis and Manuel Vargas. Deeper understonding of the homography decomposition for vision-basse control. PhD thesis, Inria, 2007.
[26] Yunze Man, Liang- Yan Gui, and Yu- Yu-Xiong Wang. BEV-Guided Multi-Modality Fusion for Driving Perception. In CVPR, 2023.
[27] Ben Mildenhall, Pratul Srinivasan, Matthew Tancik, Jonathan Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65 (1):99-1066 106, 2021. Publisher: ACM New York, NY, USA.
[28] Thomas Müller. Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphics primitives with multiresolution hash encoding, ACM Trans. Graph., 41 41(4); 102:1:00:110 2022.
[29] Julian Ost, Fahim Mannan, Nils Thucrey, Julian Knodt, and Felix Heide. Neural seene graphs for dynamic scenes. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2856 2865,286,2222 2021.
[30] F. Perazzi, J. Pont-Tuset, B. McWilliass, L. Van Gool, M. Gross, and A. Sorkine-Horunng A benchmark dataset and evaluation methodology for video object segmentation. In Computer Vision and Pattern Recognition, 2016.
[31] Albert Pumarola, Enric Corona, Gerard Pons-Moll, and Francese Moreno-Nogus D-nerf: Neural radiance fields for dynamic scenes. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10318-10327, 2021.
[32] Alex Rav-Acha, Pushmeet Kohli, Cansten Rother, and Andrew Fitzgibbon. Unwrap mosaics: new representation for video editing. ACM Transactions on Graphics (Proc. SIGGRAPH), 2733):11111; 2008.
[33] Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, Eric Mintun, Junting Pan, Kalyan Vasudev Alwala, Nicolas Carion, Chao- Yuan Wu, Ross Girshick, Piotr Dollar, and Christoph Feichtenhofer. SAM 2: Segment anything in images and videos. In The Thirteenth International Conference on Learning Representations, 2025.
[34] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical image computing and computer-asristed intervention-MICCA. 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18, pages 234-241. Springer, 2015.
[35] Colton Stearns, Adam Harley, Mikaela Uy, Florian Dubost, Federico Tombari, Gordon Wetzstein, and Leonidas Guibas. Dynamic gaussian marbles for novel view synthesis of casual monocular videos. In SIGGRAPH Asia 2024 Conference Papers, pages 1- 2024.
12

---

[36] Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard. Vijaysai Patnaik, Paul Tsui, James Guo. Yin Zhou, Yuning Chai, Benjamin Caine, et al. Scalability in perception for autonomous driving: Waymo open dataset. Proceedings of the IEEE/CVF conference un computer vision and pattern recognition. pages 2456-2454, 2020.
[37] Zachary Teed and Jia Deng. Raft: Recurrent all-pairs field transforms for optical flow. Computer Vision-ECCY 2020: 16th European Conference, Glasgow, UK. August 23-28, 2020, Proceedings, Part II 16, pages 402-419. Springer, 2020.
[38] Edgar Tretschk, Ayush Tewari, Vladislav Golyanik, Michael Zollhöfer, Christoph Lassner, and Christian Theobalt. Non-rigid neural radiance fields: Reconstruction and novel view synthesis of dynamic scene from monocular video. In Proceedings the IEEE/CVF International Conference on Computer Vision, pages 12959-12970 2021.
[39] Haithem Turki, Jason Y Zhang. Francesco Ferroni, and Deva Ramanan, Suds: Scalable urban dynamic scenes. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12375-12385, 2023.
[40] John Y. A. Wang and Edward H. Adelson. Representing moving images with layers. IEEE Transactions on Image Processing, 3(5):625-638 1994.
[41] Yuxuan Wang. Xuanyu Yi, Zike Wu, Zhao, Long Chen, and Hanwang Zhang. View-consistent 3d editing with gaussian splatting. European Conference on Computer Vision, pages 404-420. Springer, 2024.
[42] Zhou Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4):600-62,, 2004.
[43] Jiawei Yang, Boris Ivanovic, Or Litany, Xinshuo Weng, Scung Wook Kim, Boyi Li, Tong Che, Danfei Xu, Sanja Fidler, Marco Pavone, and Yue Wang. EmemeRF: Emergent spatial-temporal scene decomposition via self-supervision. In The Twelfth International Conference on Learning Representations, 2024.
[44] Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything v2. Advances in Neural Information Processing Systems, 37:21875-21911, 2024.
[45] Ze Yang, Yun Chen, Jingkang Wang, Sivabalan Manivasagam, Wci-Chiu Ma, Angi Joyce Yang, and Raquel Urtasun. Unisim: A neural closed-loop sensor simulator. In Proceedings the IEEE/CVF Conference on Computer Vision and Puttern Recognitiin,, pages 1389-1399, 2023.
[46] Vickie Ye, Zhengqi Li, Richard Tucker, Angjoo Kanazawa, and Noah Snavely. Deformable sprites for unsupervised video decomposition. Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 2657-2666, 2022.
[47] Fisher Yu, Haofeng Chen, Xin Wang, Wenqi Xian, Yingying Chen, Fangchen Liu, Vashisht Madhavan, and Trevor Darrell, Bdd100k: diverse driving dataset heterogeneous multitask learning. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pages 2636-2645, 2020.
[48] Zehao Yu, Anpei Chen, Binbin Huang. Torsten Sattler, and Andreas Geiger. Mip-splatting: Alias-free 3d gaussian splatting. In Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, pages 19447-19456, 2024.
[49] Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as perceptual metrie. In CVPR, 2018
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Claims are clearly mentioned in the abstract and introduction sections of the paper, and verified with exhaustive experiments which are further completed in our supplementary.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We clearly discuss the limitations of our method within a dedicated section.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiiatinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
14

---

Answer: [NA]
Justification:
We don't provided any theoretical results, as we verify our proposed model
empirically.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proof' in the paper should be numbered and cross referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: We discuss the experimental setup (both baselines and our method), within our supplementary and provide reproducible code for experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
15

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes)
Justification: We release our code and all data on our code page. httpp://gitthbe com/ jp- schneider/nag
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips... cc/
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ ctiaps..ccpuulic/guuu
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We discuss the dataset preparation and how the test sequences were selected well as the sub-sequence details for the experimentation.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: We evaluated our method and the baselines on a variety of challenging sequences within the Waymo Open Dataset (25 subsequences) and the DAVIS dataset (15 sequences). For each method, we report the mean performance across these sequences, along with the standard deviation within our supplementary.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
16

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out of range (c.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification; The information about the experiment setup in terms of hardware configuration, memory and time is mentioned within our supplementary material.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental rums well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code ethies
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics stttp:://nuurips.cc./.
Answer: [Yes]
Justification: Our research adheres to the NeurlPS Code Ethics. We exclusively utilized publicly available datasets, ensuring no personally identifiable information was exposed and respecting the terms of their respective licenses. Our work does not involve human subjects requiring IRB review, nor does it present foreseeable risks to safety, security, or human rights. We have carefully considered potential societal impacts and have strived to mitigate any potential for discrimination, deception, negative environmental consequences. Our focus is on advancing the field through transparent and reproducible research.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative socictal impacts of the work performed?
17

---

Answer: [NA]
Justification: While dedicated broader impact discussion section is not included in this paper, our work on Neural Atlas Graphs contributes the fundamental research in neural scene representations for dynamic environments. We believe the potential socictal impacts of our method are generally consistent with the broader implications discussed for similar research in this domain and do not foresee it leading to significantly different societal impacts compared to other advancements in this field. Our primary focus on the technical contributions and rigorous evaluation of our proposed approach.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g.. pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: Our work focuses on learning editable righuutsolution scene representations for dynamic scenes using neural atlas graphs. We do not release pre-trained language models, image generators, scraped datasets that typically carry a high risk of misuse as outlined in the question's 's examples. The primary output of our research is a novel scene representation method and its application to scene editing and understanding. Therefore, specific safeguards for responsible release of high-risk models or data are not directly applicable to our work.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
18

---

Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We have properly credited the creators and original owners of all assets used, including datasets (Waymo. DAVIS) with explicit mention and respect for their licenses. To the best of our knowledge, all existing works relevant research have been appropriately cited.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g.. website). the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperswi thcode. com datasees has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
While our primary contribution is novel method (Neural Atlas Graphs), we will be releasing the evaluation data we generated, such masks, to facilitate reproducibility and further research. These assets will be accompanied by comprehensive documentation detailing their creation process, format, and intended use.
Justification:
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: None of the mentioned performed.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
19

---

Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: None of the mentioned was performed.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing. editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: Our method and any contributions we provide do not rely on LLMs.
Guidelines:
The answer NA means that the method development in this research does involve L.MSs as any important, original, or non-standard components.
Please refer to our LLM policy (bttps: /ttpp::////rrrs..... cc/ /oo//o/rrnnee//22/5//////1// for what should or should not described.
20


Reviews:
[{"review_id": "4aesgaVqLq", "reviewer": "Reviewer_WUKh", "summary": "This paper introduces Neural Atlas Graphs (NAGs), a dynamic scene representation that combines 2D neural atlases with an object-centrist 3D scene graph. Every object and also the background region is modeled as a textured planar surface with its own view-dependent neural field, optical flow, and spline-based rigid trajectory. The model is optimized at test time using only RGB photometric loss. Experiments on Waymo and DAVIS show strong performance on reconstruction metrics and demonstrate compelling view-consistent editing capabilities, including object removal, appearance changes, and motion modifications.", "questions": "Weaknesses\n(1). The paper does not evaluate temporal consistency quantitatively (https://arxiv.org/abs/2506.01004 ), despite being a video reconstruction and editing method. A metric such as Temporal-LPIPS or inter-frame variance is necessary to support claims of stability.\n\n(2). The evaluation is limited to Waymo and DAVIS, which are fairly structured datasets. The evaluations seem to have been done while the frame of reference is stationary - can the authors consider evaluating on datasets where the frame of reference is moving. Given it is a video based method and movement could be on the side of object as well as reference?\n\n(3). How can the motion of the frame of reference be handled while editing a scene - such that edits are reasonably accurate and clean\n\n(4). The editing results are purely qualitative. A stronger evaluation of performance would allow to fairly ascertain the quality of the results with confidence. Below are metrics, some of which can be used to asses the method- \n    Fréchet Video Distance (FVD), Learned Perceptual Image Patch Similarity (Though it is an image-based metric, it could be be     applied frame-by-frame/or on keyframes of the edited video to assess quality and realism of the changes introduced)\n\n   Peak Signal-to-Noise Ratio,SSIM: for evaluating specific edited regions for a comparison.\n       \n   Temporal-LPIPS - discussed in point 1", "limitations": "see Questions.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "Strengths\n(1) The paper is clearly written and easy to follow, with a strong motivation.\n(2) As can be seen in the supplementary video, the method effectively supports high-resolution, temporally coherent video edits using an intuitive object-wise plane representation.\n(3) Strong qualitative results show convincing edits across multiple scenes and domains.\n(4) The design is supported by well-structured ablation studies showing the contribution of flow, view-dependence, and trajectory modeling.", "was_revised": true, "final_justification": "The responses by the authors are satisfactory. I find the work useful across multiple areas and settings in autonomous driving. I have requested the reviewers to introduce a section regarding regarding the analysis when both the object and the frame of reference are in motion and also add the code for this use-case in the final code.", "rebuttal": "We appreciate the thorough and constructive feedback provided by reviewer WUKh. \n\n**1. Quantitative Evaluation of Temporal Consistency**\n\nThe reviewer suggested evaluating temporal consistency with metrics such as Temporal-LPIPS or Fréchet Video Distance (FVD). We report these scores in the following and will include them in the revised work, as we agree that this strengthens the overall quantitative evaluation:\n\n| (Dataset) Method | FVD ↓ | T-LPIPS |\n|---|---:|---:|\n| (W) Ours | 174 ± 238 | 0.063 ±  0.031 |\n| (W) ORE [1] | 423 ± 446 | 0.055 ± 0.029 |\n| (W) ERF [2] | 439 ± 350 | 0.053 ± 0.026 |\n| (W) GT | N/A | 0.079 ± 0.029 |\n| (D) Ours | 108 ±  35 | 0.143 ± 0.022 |\n| (D) ORF [4] | 986 ± 153 | 0.104 ± 0.023 |\n| (D) LNA [3] | 595 ±  38 | 0.116 ± 0.022 |\n| (D) GT | N/A | 0.155 ± 0.020 |\n\nIn the above table, we report the FVD scores as the mean over the evaluated sequences and the Temporal (frame-by-frame) LPIPS, including their respective standard deviation. The scores have been calculated on the ablation sequences for Waymo (W) and blackswan, bear, and boat sequences for Davis (D). For FVD, lowest is best. We note that T-LPIPS is not a quality but rather a consistency metric, given its frame-by-frame application, and report the ground truth (GT) here as comparison. We find that our model outperforms baselines on FVD, for comparable image consistency scores, which aligns with the claims of the main work.\n\nWe would also like to clarify that we computed the inter-frame standard deviation and reported the findings in the supplementary material (Tables 2, 3, and 4) due to space constraints in the main paper. For instance, the inter-frame standard deviation of PSNR for our method is very competitive at ±1.15 (mean PSNR 41.85), compared to OmniRe's ±2.31 (mean PSNR 36.92) and EmerNeRF's ±0.65 (mean PSNR 34.93). We will highlight these statistics in the revised main text, and further emphasize the importance of temporal consistency in addition to individual frame fidelity.\n\n**2. Evaluation on Stronger Ego Motion**\n\nThe DAVIS dataset has more camera motion than the presented Waymo sequences, but does not contain the high rotational ego motion of some autonomous driving scenes. As suggested, we add an ablation on this category of scene below:\n\n| Obj | Vehicle PSNR |  |  | Vehicle SSIM |  |  | Human PSNR |  |  | Human SSIM |  |  |\n|---: |---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|\n| Seq. | Ours | ORE [1] | ERF [2] | Ours | ORE [1] | ERF [2] | Ours | ORE [1] | ERF [2] | Ours | ORE [1] | ERF [2] |\n| 191 | 37.60 | 30.95 | 25.81 | 0.959 | 0.936 | 0.750 | 34.40 | 25.93 | 22.91 | 0.908 | 0.742 | 0.707 |\n| 254 | 35.07 | 30.06 | 27.05 | 0.926 | 0.923 | 0.786 | 34.42 | 26.46 | 23.29 | 0.919 | 0.863 | 0.703 |\n\n| Avg | PSNR |  |  | SSIM |  |  | LPIPS |  |  |\n|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|\n| Seq. | Ours | ORE [1] | ERF [2] | Ours | ORE [1] | ERF [2] | Ours | ORE [1] | ERF [2] |\n| 191 | 32.02 | 33.22 | 29.98 | 0.892 | 0.950 | 0.872 | 0.209 | 0.090 | 0.244 |\n| 254 | 31.70 | 31.86 | 29.17 | 0.911 | 0.951 | 0.872 | 0.190 | 0.095 | 0.247 |\n\nFor foreground objects, managed by dedicated nodes, our approach achieves substantial improvements (up to 8 PSNR for Vehicles/Humans) compared to all baselines. This is primarily attributed to our robust rigid motion model, which, being independent of global camera motion, is also well-suited for large ego-motion environments. The overall image scores are comparable to OmniRe and EmerNeRF (second table). As mentioned in the Limitations section, this score is not substantially higher, as we observe more artifacts in the background, as this large motion has to be generated through learning large flow vectors based on photometric loss (rather than from explicit 3D projective geometry).\n\nWe emphasize that our proposed Neural Atlas Graph Model is fundamentally “2.5D” (planes + flow), with a large inductive bias which helps regularize solutions of low-parallax / sparsely observed scene elements – e.g., a vehicle that quickly drives through the scene while we only observe one side of it – but is not as well suited to model complex 3D structures with large amounts of self-occlusion. However, we believe our findings in the work indicate that this is an interesting direction for future research, and hope our work can be extended as a vehicle/object-centric graph model (combining both 2D and 3D primitives in one render graph). We will add the evaluation of these sequences into the revised work, along with a discussion on this direction of future research.\n\n\nGiven that we are not allowed to upload visual content in the rebuttal given the newest NeurIPS policies, we include a short text description of the added scenes:  \n\nThe added sequences s-1918764220984209654 (191) are showing a drivepath into a suburban pedestrian crossing, where the ego-vehicle stops shortly, vehicles and a pedestrian are crossing, after which the ego vehicle drives further into the scene.\n\nThe sequence s-2547899409721197155 (254) shows the ego vehicle also on a pedestrian crossing, where a pedestrian with stroller and vehicles passing by, after which the car performs a right turn and drives into the scene, faced by a streetcar.\n\n**3. Editing under Motion**  \n\nThe reviewer raised a question regarding how camera motion is handled during scene editing to ensure accurate and clean edits. Edits are projected back to the planar nodes based on one or multiple edited frames, forming an **\"overlay-layer\" on the object plane (not the image plane),** which is therefore largely independent of the camera's ego and object-motion. However, they *are* dependent on the viewing angle with respect to the object. Our coarse-to-fine strategy aims to learn most of the view-dependent transformations in the planar flow field, which also affects the overlay-layer, resulting in deformations that can be perceived as view-consistent editing. For example, see Fig. 4., Sup. Fig. 8, 9, or the corresponding videos in the supplementary. In the boat and blackswan scene (Sup. Fig. 8, 9), the camera and object are moving independently, yet show a visually consistent texture edit. \n\nThe frame of reference is used to enable intuitive editing in image space by robustly projecting edits to the planar domain via numerical inversion of the learned flow. This significantly simplifies the editing process compared to direct planar manipulation (as in [3]) and allows seamless integration with off-the-shelf image editing tools and improves editing comparability by making the texture flow-independent. We will further clarify this process in the revised work.\n\n**4. Quantitative Scores for Editing**  \n\nEstablishing ground truth for arbitrary edited frames on real data is challenging, making the quantitative evaluation of (texture) editing quality particularly difficult. This is a common issue in the field, and we observe that existing baseline methods [3, 4] also primarily rely on qualitative assessments. Nevertheless, we evaluated the Temporal (frame-by-frame) LPIPS and Fréchet inception distance (FID) [5] metrics to quantify the consistency and similarity of the decomposed objects, or edited frames versus their succeeding frames, for all our provided editing figures. We find that the consistency is comparable to the ground truth (GT).\n\n|  | T-LPIPS |  |  | FID [5] |  |  |\n|:---|---:|---:|---:|---:|---:|---:|\n| Edit | Ours | ORE  [1] | GT | Ours | ORE  [1] | GT |\n| Fig. 3, Seg. 125 | 0,052 ± 0,027 | 0,054 ± 0,032 | 0,081 ± 0,033 | 2.244 ± 1.857 | 1.972 ± 1.326 | 2.228 ± 1.578 |\n| Fig. 3, Seg. 141 | 0.056 ± 0.048 | 0.063 ± 0.062 | 0.103 ± 0.072 | 0.999 ± 1.032 | 1.111 ± 1.308 | 1.517 ± 1.222 |\n| Figure 4 | 0.037 ± 0.006 | N/A | 0.043 ± 0.006 | 0.175  | N/A | 0.173 |\n| Supl. Fig. 8 | 0.249 ± 0.028 | N/A | 0.261 ± 0.026 | 0.255 | N/A | 0.185 |\n| Supl. Fig. 9 | 0.058 ± 0.012 | N/A | 0.080 ± 0.012 | 0.132 | N/A | 0.180 |\n\nFor T-LPIPS we report the mean and standard deviation over all image-pairs and objects in case of Fig. 3. For FID we report the standard deviation only for the per-object decomposition evaluation, given that it is computed as a distributional measure. For all stated edits, our consistency results are comparable to our reference method or the respective ground truth (GT).\n\n**References**\n\n[1] Chen, Ziyu, et al. \"OmniRe: Omni Urban Scene Reconstruction.\" The Thirteenth International Conference on Learning Representations. 2025\n\n[2] Yang, Jiawei, et al. \"EmerNeRF: Emergent spatial-temporal scene decomposition via self-supervision.\" arXiv preprint arXiv:2311.02077 (2023).  \n\n[3] Kasten, Yoni, et al. \"Layered neural atlases for consistent video editing.\" ACM Transactions on Graphics (TOG) 40.6 (2021): 1-12.  \n\n[4] Lin, Geng, et al. \"Omnimatterf: Robust omnimatte with 3d background modeling.\" Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.  \n\n[5] Heusel, Martin, et al. \"Gans trained by a two time-scale update rule converge to a local nash equilibrium.\" *Advances in neural information processing systems* 30 (2017)."}, {"review_id": "7bWgAv1VL6", "reviewer": "Reviewer_HUMq", "summary": "This paper proposes Neural Atlas Graphs (NAGs), a novel hybrid scene representation designed for editing dynamic scenes. Unlike existing approaches that often compromise on either editability or the ability to model complex scenes, NAGs combine the strengths of both by representing scenes as a graph of moving 3D planes, with each plane corresponding to an object or the background. This representation facilitates precise 2D appearance editing as well as 3D positioning and ordering of scene elements, enabling flexible and intuitive scene manipulation. NAGs deliver state-of-the-art quantitative performance on the Waymo Open Dataset for driving scenes. Furthermore, the method demonstrates strong generalization beyond automotive contexts, achieving impressive matting and video editing results on the DAVIS dataset, which includes a diverse set of human- and animal-centric scenes.", "questions": "1. How does the proposed NAG approach perform when processing video inputs with significant camera motion?\n\n2. What is the capability of NAGs for novel view synthesis with large camera displacement?\n\n3. Can the authors explain why training NAGs for a single scene takes so long?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "### Strengths\n1. NAGs model dynamic scenes as a graph of moving 3D planes, with each plane representing an individual object or the background. This approach effectively balances the ability to handle complex, dynamic scenes with high editability, allowing for both 2D appearance modifications and 3D spatial adjustments. \n\n3/ The qualitative results presented demonstrate robust reconstruction, decomposition, and editing of dynamic scenes, preserving scene details and enabling fine-grained edits.\n\n### Weaknesses\n1.  While NAGs provide strong editability, the practical significance of this feature in autonomous driving scenarios is uncertain. Compared to NeRF/3DGS-based approaches, NAGs’ hybrid approach may not offer a clear advantage for applications where consistent 3D rendering is critical.\n\n2. The qualitative results lack significant camera motion, with no demonstration of novel view synthesis capabilities. This raises concerns about the method’s applicability in scenarios requiring dynamic camera perspectives. If novel view synthesis is not a priority, traditional video editing models might suffice, potentially diminishing the need for NAGs in certain use cases.\n\n3.  The experimental evaluation omits comparisons with established image or video editing methods, particularly in tasks like matting or appearance editing where traditional methods may already perform effectively.\n\n4. Training NAGs for a single scene requires 2 to 6 hours on an NVIDIA L40 GPU. This significant training time may pose a barrier to practical adoption.", "was_revised": true, "final_justification": "After reading the authors' rebuttal and other reviewers' opinions, most of my concerns have been addressed, and I would like to keep my positive rating.", "rebuttal": "We thank reviewer HUMq for their analysis of and feedback on our work.\n\n**1. Practical Significance in Autonomous Driving and Comparison to NeRF/3DGS**\n\nThe reviewer raises a point regarding the practical significance of Neural Atlas Graphs (NAG) editability in autonomous driving scenarios, especially compared to NeRF/3DGS-based approaches, which prioritize consistent 3D rendering. Our intent with NAGs is to offer a novel hybrid representation that excels in intuitive, object-centric manipulation, which is distinct from purely volumetric or splatting-based 3D reconstruction. While NeRF/3DGS methods excel at novel view synthesis and high-fidelity rendering from dense captures, their *texture editability is challenging and often requires re-training or complex parameter adjustments to the implicit volume or point cloud*, especially for complex non-rigid motion and semantic object manipulation.\n\nNAGs, by explicitly decomposing the scene into editable planar objects, offer a unique advantage in applications requiring semantic understanding and direct editing of texture.\n\nFor a concrete application example in autonomous driving, we demonstrated in Figure 5 (supplementary material) a texture editing where we retextured the street to include a speed limit sign. As this texture editing interacts realistically with dynamic objects in the scene, forming realistic occlusions, such a technique could be highly valuable for generating realistic data for driving simulators or for training automotive image recognition agents. This demonstrates a practical application where explicit object manipulation and view-consistent texture changes are essential, complementing, rather than directly replacing existing 3D methods.\n\nWe will improve the manuscript to clarify this.\n\n**2. Evaluation on Stronger Ego Motion / Question 1.**\n\nThe DAVIS dataset has more camera motion than the presented Waymo sequences, but does not contain the high rotational ego motion of some autonomous driving scenes. As suggested, we add an ablation on this category of scenes below:\n\n| Obj | Vehicle PSNR |  |  | Vehicle SSIM |  |  | Human PSNR |  |  | Human SSIM |  |  |\n|-:|-:|-:|-:|-:|-:|-:|-:|-:|-:|-:|-:|-:|\n| Seq. | Ours | ORE [1] | ERF [2] | Ours | ORE [1] | ERF [2] | Ours | ORE [1] | ERF [2] | Ours | ORE [1] | ERF [2] |\n| 191 | 37.60 | 30.95 | 25.81 | 0.959 | 0.936 | 0.750 | 34.40 | 25.93 | 22.91 | 0.908 | 0.742 | 0.707 |\n| 254 | 35.07 | 30.06 | 27.05 | 0.926 | 0.923 | 0.786 | 34.42 | 26.46 | 23.29 | 0.919 | 0.863 | 0.703 |\n\n|Avg|PSNR|||SSIM|||LPIPS|||\n|-:|-:|-:|-:|-:|-:|-:|-:|-:|-:|\n|Seq.|Ours|ORE [1]|ERF [2]|Ours|ORE [1]|ERF [2]|Ours|ORE [1]|ERF [2]|\n|191|32.02|33.22|29.98|0.892|0.950|0.872|0.209|0.090|0.244|\n|254|31.70|31.86|29.17|0.911|0.951|0.872|0.190|0.095|0.247|\n\nFor foreground objects, managed by dedicated nodes, our approach achieves substantial improvements (up to 8 PSNR for Vehicles/Humans) compared to all baselines. This is primarily attributed to our robust rigid motion model, which, being independent of global camera motion, is also well-suited for large ego-motion environments. The overall image scores are comparable to OmniRe (ORE) and EmerNeRF (ERF) (second table). As mentioned in the Limitations section, this score is not substantially higher, as we observe more artifacts in the background, as this large motion has to be generated through learning large flow vectors based on photometric loss (rather than from explicit 3D projective geometry).\n\nWe emphasize that our proposed NAG Model is fundamentally “2.5D” (planes + flow), with a large inductive bias which helps regularize solutions of low-parallax / sparsely observed scene elements – e.g., a vehicle that quickly drives through the scene while we only observe one side of it – but is not as well suited to model complex 3D structures with large amounts of self-occlusion. However, we believe our findings in the work indicate that this is an interesting direction for future research, and hope our work can be extended as a vehicle/object-centric graph model (combining both 2D and 3D primitives in one render graph).\n\nWe will add the evaluation of the newly introduced sequences into the revised work, along with a discussion on this direction of future research.\n\nAs pointed out by the reviewer, our main priority is not novel view synthesis, but rather in the video editing, which we address in the following point 3.\n\n**3. Comparison to Video Editing Methods**\n\nWe note, *that we did compare our method to established video matting methods* on the DAVIS dataset. Specifically, in Table 3 of our paper, our NAG demonstrates clear performance gains (7.3 dB PSNR improvement) over these video editing baselines (Layered Neural Atlases [3] (LNA) - most recent texture editing, OmnimatteRF [4] (ORF) - most recent matting), validating our ability to achieve robust decomposition and reconstruction even in the absence of 3D supervision, while simultaneously enabling flexible editing. This evaluation also confirms that even without a primary focus on novel view synthesis, NAGs offer advantages over traditional video editing approaches in terms of consistency, decomposition and editing.\n\nTo make this clearer, we will add a paragraph in the revised work, elaborating on these recent video matting methods and their usage. Yet, we remain open to further baseline suggestions for our revised work.\n\n**4. Training Time / Question 3.**\n\nThe reviewer finds the training time of 2 to 6 hours per scene on an NVIDIA L40 GPU as a potential barrier to practical adoption. We agree that shorter training times are highly desirable for practical applications. However, we would like to contextualize these times within the field of dynamic neural scene representation. Other state-of-the-art methods, such as OmniRe [1], also report comparable training durations (e.g., approximately one hour for a single scene at 960x640 resolution), higher on larger resolutions, while the proposed method runs for *frames with 4x more pixels.*\n\nFurthermore, highly optimized methods like Gaussian Splatting, while achieving impressive speeds, have benefited from dedicated native CUDA implementations and extensive optimization efforts over recent years [5, 6]. Below, we provide observed training times for the full resolution (1920 x 1280 for Waymo and 1920 x 1080 for DAVIS) in all experiments:\n\n| (Dataset) Method | Training Time (Minutes) |\n|-|-:|\n| (W) Ours | 140 |\n| (W) ORE [1] | 127 |\n| (W) ERF [2] | 42 |\n| (D) Ours | 198 |\n| (D) ORF [4] | 279 |\n| (D) LNA [3] | 444 |\n\nFor Waymo (W), we used a subsequence from 975 with a length of 68 frames, and for DAVIS the bear sequence with 74 frames in their respective resolution. Our training time is significantly shorter than the other atlas methods on DAVIS (D). On Waymo, we are slower than the OmniRe (ORE) [1] given its efficiency in rendering. EmerNeRF (ERF) [2] is faster, given its not a scene graph method, and does not have a dedicated model per object.\n\nBelow, we elaborate on three key components that contribute to our training time while being essential for a high-quality and editable NAG representation:\n\n1. Extensive Ray-Casting: Necessary for high-resolution (up to 1920x1280), dynamic video learning, training involves extensive ray-casting (2.8×10$^8$ rays/epoch, 80 epochs). Time reduces for lower resolutions/shorter videos (<20 min).\n2. Multi-Stage Optimization: A three-phase strategy ensures stable convergence and enhanced texture editability.\n3. Per-Object Networks: Training time scales with scene complexity due to dedicated per-object/background neural networks.\n\nAblation studies on different parameterizations and full training details are available in the Experimental Details (Sec. 2) of our supplementary material.\n\n**Questions**\n\n**Q1:**  \nSee above (\"2. Evaluation on Stronger Ego Motion / Question 1.\")\n\n**Q2:**  \nAs a *2.5D method*, NAGs are designed to infer scene structure and motion primarily from 2D observations. Consequently, novel view synthesis capabilities are inherently bound to the range of view angles seen during the fitting process. While smaller angular and positional variations of objects or the camera are possible and typically yield good results (see Fig. 4 in Supplementary Material), very large camera displacements that are significantly 'out-of-domain' of the original video cannot be handled adequately. This is because our atlas nodes are planar in 3D space, and they mimic depth parallax implicitly via flow and view-dependent fields. Therefore, large view changes can lead to perspective errors, such as looking orthogonally onto plane edges or causing objects to disappear. This limitation is inherent to our design choice: we modeled the method as 2.5D to facilitate texture edits, rather than prioritizing novel view synthesis.\n\n**Q3:**   \nSee above (\"4. Training Time / Question 3.\")\n\n**References**\n\n[1] Chen, Ziyu, et al. \"OmniRe: Omni Urban Scene Reconstruction.\" The Thirteenth International Conference on Learning Representations. 2025\n\n[2] Yang, Jiawei, et al. \"EmerNeRF: Emergent spatial-temporal scene decomposition via self-supervision.\" arXiv preprint arXiv:2311.02077 (2023).\n\n[3] Kasten, Yoni, et al. \"Layered neural atlases for consistent video editing.\" ACM Transactions on Graphics (TOG) 40.6 (2021): 1-12.\n\n[4] Lin, Geng, et al. \"Omnimatterf: Robust omnimatte with 3d background modeling.\" Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.\n\n[5] Kerbl, Bernhard, et al. \"3D Gaussian splatting for real-time radiance field rendering.\" ACM Trans. Graph. 42.4 (2023): 139-1.\n\n[6] Wu, Guanjun, et al. \"4d gaussian splatting for real-time dynamic scene rendering.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2024."}, {"review_id": "pODyLDG0F6", "reviewer": "Reviewer_kRsQ", "summary": "The paper investigates the problem of learning editable scene representations. The authors proposes a method, called Neural Atlas Graph (NAG), that combines the strengths of two prominent methods, neural scene graphs and neural atlases. The proposed method takes a video scene and the masks of foreground objects as an input, and assigns a node to each foreground object and a separate node for the background. The nodes contain information of the color, opacity and position and can model the rigid and non-rigid transformations of objects. NAG objective consists of an $\\ell_1$ term for ground truth and rendered color predictions and a term that encourages the predicted opacity to match the provided coarse masks. The editing is implemented similar to the neural atlases. The authors evaluate their method for scene reconstruction and video editing. The quantitative results show significant improvements on the baselines.", "questions": "In general, additional experiments as explained in the quality section would be helpful. Additionally,\n1. How much does the method rely on the coarse masks $\\mathcal{M}$? How sensitive is the estimation of $s_i \\in \\mathcal{N}_i$ to the coarse masks?\n2. The scenes with small ego motion were chosen for experiments. How much the output quality of the proposed and baseline methods deteriorate in scenes with higher ego motion? Could you provide sample scenes for such cases?\n3. How does editing quality compare to the prior work? While the editing quality is impressive in provided videos, it would be helpful to compare it with the existing methods.", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 4, "originality": 3, "strengths_and_weaknesses": "**Quality:** This paper presents a strong and methodologically sound contribution. The proposed method is clearly explained, with detailed experimental evaluations that effectively demonstrate its capabilities. The evaluation setups are transparent and reproducible, and the authors provide an anonymous codebase for reproducibility. The weaknesses could be listed as follows:\n1. The number of baselines considered is relatively low compared to prior work. For instance, baseline [1] includes five different comparison methods, while this paper provides only two. Including a broader set of baselines would strengthen the empirical support. Furthermore, a brief explanation of the baseline methods—potentially in the appendix, as done in [1]—would enhance clarity and completeness.\n2. The paper does not include comparisons of editing results with prior work. Baseline [1], for example, provides such results, which could offer a meaningful point of comparison.\n3. The evaluation would benefit from inclusion of more challenging scenes, such as those with high ego-motion or fast-moving objects, similar to what is explored in [1] (Section D.3). This would provide additional evidence of the method’s robustness and generalizability.\n\n**Clarity:** The paper is well-written and clearly presented. As a suggestion, adding a comparison between neural atlases, neural scene graphs, and neural atlas graphs (NAGs) could enhance the paper’s self-containment and help clarify the key ideas and design choices.\n\n**Significance and Originality:** While the paper combines two methods in the literature, this novel fusion yields impressive results both qualitatively and quantitatively.\n\n[1] Ziyu Chen, Jiawei Yang, Jiahui Huang, Riccardo de Lutio, Janick Martinez Esturo, Boris Ivanovic, Or Litany, Zan Gojcic, Sanja Fidler, Marco Pavone, Li Song, and Yue Wang. Omnire: Omni urban scene reconstruction. In The Thirteenth International Conference on Learning Representations, 2025.", "was_revised": true, "final_justification": "The authors propose an interesting method. The empirical results show significant improvements over the baselines. The paper is well-written and the authors will add additional discussion that will enhance the readability further. The authors' rebuttal was extensive and answered the concerns. The authors are also clear about the main limitations of their work.", "rebuttal": "We sincerely thank the reviewer for the thoughtful and detailed review.\n\n**1. Number of Baselines**  \n\nThe reviewer considered the number of baselines low compared to prior work. We would like to clarify that we actually compared our method against four distinct baselines: OmniRe [1] (a Gaussian Splatting Method), EmerNeRF [2] (a Dynamic NeRF Method), Layered Neural Atlases [3] (a texture editing-capable method), and OmnimatteRF [4] (a recent matting method). Note that we *selected baselines for which code was available* and that ideally supported per-object removal, allowing for meaningful comparisons in our hybrid setting. We find that comparing across these diverse fields (Gaussian Splatting, Dynamic NeRF, Texture Editing, and Matting) provides strong empirical support for the broad applicability and competitive performance of NAGs, rather than limiting the comparison to a single sub-domain. Given that OmniRe demonstrated improved performance compared to the other five Gaussian Splatting methods, we focused our comparisons on it rather than including additional similar approaches. We will include a brief explanation of these baseline methods in the appendix to enhance clarity and completeness, similar to [1] and add additional automotive baselines within the revised version of our work.\n\n**2. Comparison of Editing Results to Prior Work / Question 3**\n\nThe reviewer suggested including comparisons of editing results with prior work, specifically mentioning [1]. We do not find that OmniRe [1] states direct *editing* comparisons against baselines. To our knowledge, neither their main paper nor supplementary material explicitly presents quantitative or qualitative comparisons of *edited* scenes against other methods. While they do show their own scene decomposition in Figure 1 and a qualitative result comparison of the complete scene in Figure 4, these do not allow for a direct comparison of editing capabilities against baselines.\n\nIn our paper, we explicitly compare the visual results of scene decomposition in Figure 3 against OmniRe on both a populated car scene and an adverse weather scene, confirming that our method excels at extracting objects and produces fewer artifacts. Furthermore, we demonstrate our scene decomposition and visual quality on two DAVIS dataset scenes in Supplementary Figure 6 (and the supplementary videos) compared against recent video editing and matting methods, again showing superior visual quality.  \\\nThe inherent difficulty in finding ground truth for arbitrary semantic edits (e.g., changing a car's color, adding a speed limit sign) makes direct quantitative comparisons challenging for any method in this domain.   \nWhile we acknowledge this challenge, we have computed the Temporal (frame-by-frame) LPIPS and Fréchet inception distance (FID) [5] for our editing figures to quantify their temporal consistency and overall perceptual impact, providing an objective measure where possible. We find that the consistency is comparable to the ground truth (GT) or respective baselines.\n\n|  | T-LPIPS |  |  | FID [5] |  |  |\n|---|---:|---:|---:|---:|---:|---:|\n| Edit | Ours | ORE  [1] | GT | Ours | ORE  [1] | GT |\n| Fig. 3, Seg. 125 | 0,052 ± 0,027 | 0,054 ± 0,032 | 0,081 ± 0,033 | 2.244 ± 1.857 | 1.972 ± 1.326 | 2.228 ± 1.578 |\n| Fig. 3, Seg. 141 | 0.056 ± 0.048 | 0.063 ± 0.062 | 0.103 ± 0.072 | 0.999 ± 1.032 | 1.111 ± 1.308 | 1.517 ± 1.222 |\n| Figure 4 | 0.037 ± 0.006 | N/A | 0.043 ± 0.006 | 0.175  | N/A | 0.173 |\n| Supl. Fig. 8 | 0.249 ± 0.028 | N/A | 0.261 ± 0.026 | 0.255 | N/A | 0.185 |\n| Supl. Fig. 9 | 0.058 ± 0.012 | N/A | 0.080 ± 0.012 | 0.132 | N/A | 0.180 |\n\nFor T-LPIPS we report the mean and standard deviation over all image-pairs and objects in case of Fig. 3. For FID we report the standard deviation only for the per-object decomposition evaluation, given that it is computed as a distributional measure.\n\n**3. Evaluation on Stronger Ego Motion / Question 2.**\n\nThe DAVIS dataset has more camera motion than the presented Waymo sequences, but does not contain the high rotational ego motion of some autonomous driving scenes. As suggested, we add an ablation on this category of scene below:\n\n| Obj | Vehicle PSNR |  |  | Vehicle SSIM |  |  | Human PSNR |  |  | Human SSIM |  |  |\n|---: |---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|\n| Seq. | Ours | ORE [1] | ERF [2] | Ours | ORE [1] | ERF [2] | Ours | ORE [1] | ERF [2] | Ours | ORE [1] | ERF [2] |\n| 191 | 37.60 | 30.95 | 25.81 | 0.959 | 0.936 | 0.750 | 34.40 | 25.93 | 22.91 | 0.908 | 0.742 | 0.707 |\n| 254 | 35.07 | 30.06 | 27.05 | 0.926 | 0.923 | 0.786 | 34.42 | 26.46 | 23.29 | 0.919 | 0.863 | 0.703 |\n\n| Avg | PSNR |  |  | SSIM |  |  | LPIPS |  |  |\n|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|\n| Seq. | Ours | ORE [1] | ERF [2] | Ours | ORE [1] | ERF [2] | Ours | ORE [1] | ERF [2] |\n| 191 | 32.02 | 33.22 | 29.98 | 0.892 | 0.950 | 0.872 | 0.209 | 0.090 | 0.244 |\n| 254 | 31.70 | 31.86 | 29.17 | 0.911 | 0.951 | 0.872 | 0.190 | 0.095 | 0.247 |\n\nFor foreground objects, managed by dedicated nodes, our approach achieves substantial improvements (up to 8 PSNR for Vehicles/Humans) compared to all baselines. This is primarily attributed to our robust rigid motion model, which, being independent of global camera motion, is also well-suited for large ego-motion environments. The overall image scores are comparable to OmniRe and EmerNeRF (second table). As mentioned in the Limitations section, this score is not substantially higher, as we observe more artifacts in the background, as this large motion has to be generated through learning large flow vectors based on photometric loss (rather than from explicit 3D projective geometry).\n\nWe emphasize that our proposed Neural Atlas Graph Model is fundamentally “2.5D” (planes + flow), with a large inductive bias which helps regularize solutions of low-parallax / sparsely observed scene elements – e.g., a vehicle that quickly drives through the scene while we only observe one side of it – but is not as well suited to model large 3D structures with large amounts of self-occlusion. However, we believe our findings in the work indicate that this is an interesting direction for future research, and hope our work can be extended as a vehicle/object-centric graph model (combining both 2D and 3D primitives in one render graph).\n\nWe will add a discussion for the newly added evaluation sequences to the revised work, along with a discussion on this direction of future research.\n\nGiven that we are not allowed to upload visual content in the rebuttal given the newest NeurIPS policies, we include a short text description of the added scenes:\n\nThe added sequences s-1918764220984209654 (191) are showing a drivepath into a suburban pedestrian crossing, where the ego-vehicle stops shortly, vehicles and a pedestrian are crossing, after which the ego vehicle drives further into the scene.\n\nThe sequence s-2547899409721197155 (254) shows the ego vehicle also on a pedestrian crossing, where a pedestrian with stroller and vehicles passing by, after which the car performs a right turn and drives into the scene, faced by a streetcar.\n\n**4. Reliance on Coarse Masks & Sensitivity**\n\nThe reviewer inquired about the method's reliance on and sensitivity to coarse masks. We initialize each node's scale using the largest mask observed across all frames, incorporating an empirically found 80% margin. This margin is specifically designed to encompass object-associated effects such as shadows and reflections, which typically extend beyond the immediate object silhouette and are of low occupancy. The approach preconditions the node's spatial extent, but ensures only weak reliance on the pixel-level quality of individual frame masks for precise sizing. To further test robustness to varying input mask qualities, we will conduct experiments on perturbed masks for the final version.\n\n**5. Clarity Suggestion**\n\nThe reviewer suggested adding a comparison between neural atlases, neural scene graphs, and NAGs to enhance clarity and self-containment. We will do so and add a discussion of the key ideas, design choices, strengths, and weaknesses of (Layered) Neural Atlases, Neural Scene Graphs, and Neural Atlas Graphs (NAGs).\n\n**References**  \n\n[1] Chen, Ziyu, et al. \"OmniRe: Omni Urban Scene Reconstruction.\" The Thirteenth International Conference on Learning Representations. 2025 \n\n[2] Yang, Jiawei, et al. \"EmerNeRF: Emergent spatial-temporal scene decomposition via self-supervision.\" arXiv preprint arXiv:2311.02077 (2023).\n\n[3] Kasten, Yoni, et al. \"Layered neural atlases for consistent video editing.\" ACM Transactions on Graphics (TOG) 40.6 (2021): 1-12. \n\n[4] Lin, Geng, et al. \"Omnimatterf: Robust omnimatte with 3d background modeling.\" Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.\n\n[5] Heusel, Martin, et al. \"Gans trained by a two time-scale update rule converge to a local nash equilibrium.\" *Advances in neural information processing systems* 30 (2017)."}, {"review_id": "qan76Btx2M", "reviewer": "Reviewer_4LSW", "summary": "This work introduces a hybrid dynamic scene representation method called Neural Atlas Graph (NAG). It combines the advantages of two major approaches: neural scene graphs and neural atlas representations. NAG comprises nodes similar to scene graphs, but each node is represented by a neural atlas, making it more editable while maintaining visual consistency.", "questions": "If the authors clearly clarify the originality of the method in the relevant section and include comparisons with a SOTA atlas-based method, I would consider increasing the originality score.\n\nIf the authors answer the following questions, I would consider increasing the clarity score:\n\n•\tIs the camera pose initialized using COLMAP or another approach?\n\n•\tAre both static and dynamic objects used during initialization?\n\n•\tAre all components of a node (c_i, $\\alpha_i, f_i, g_i, s_i$) optimized simultaneously according to Eq. (8), or is the rigid plane pose estimated first, followed by optimization of the other components?\n\n•\tWhy are both $C_i$ (base color field) and $F_{i,c}$ (view-agnostic fields) needed? According to Eq. (5), the optimization process cannot distinguish between these two functions. Is F a learnable function while $C_i$ is not? There is no explanation of how the neural atlas map is initialized.", "limitations": "Yes.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- The overall performance surpasses other state-of-the-art (SOTA) methods.\n- The idea is compelling, and the authors provide both mathematical details and sufficient supplementary materials.\n\nWeaknesses:\n- It is difficult to distinguish the contributions of this paper from prior work in the method section. In particular, the reviewer was confused about whether each equation is derived from previous work or newly proposed. Is the method simply a combination of existing approaches, or does it include novel contributions in the integration?\n- The overall pipeline and key training details are missing, making it hard to understand the training strategy. (See questions below.)\n- Although the paper combines two approaches, the experiments only include comparisons with radiance field-based methods (e.g., ORE, ERF), not with atlas-based methods.", "was_revised": true, "final_justification": "The authors provide comprehensive implementation details to ensure full reproducibility. Rather than merely merging two existing methods, they propose a novel framework that harnesses the strengths of both. These contributions are substantial and justify accepting the paper.", "rebuttal": "We appreciate the reviewer's comprehensive and constructive assessment of our paper.\n\n**1. Distinguishing Originality**  \n\nThe reviewer sought clarification on the originality of our method, specifically asking whether it is simply a combination of existing approaches or includes novel contributions, and requested originalty details on stated equations.\n\nWe introduce NAGs as a **novel hybrid scene representation method**, positioned between Neural Scene Graphs (objects collectively form a scene graph structure) and Neural Atlases. The new representation yields an editable atlas specifically designed for video editing. While our method is inspired by various concepts in the field, our core contribution is a new representation and learning method (see Q2-Q4 below): to the best of our knowledge, there does not exist a method that models dynamic 3D scenes using 2D planar fields placed on 3D rigid motion paths in the manner we propose.\n\n**To explicitly clarify, if parts of equations are derived from others in the literature, we have explicitly cited their source directly along the equation:**\n\n- **Eq. 1:** Describes the color and alpha formation for our novel method's rendering process.\n\n- **Eq. 2:** Represents known alpha-compositing, which is standard in neural rendering (e.g., NeRFs and 3DGS), and we have explicitly cited the work of [17] for reference.\n\n- **Eq. 3 and 4:** Describe our dedicated spline-based rotation model for objects. While inspired by [6], [6] primarily focused on static binary 2D layer separation and modeled smooth camera movements, whereas our formulation is specifically adapted for per-object dynamic rigid transformations.\n\n- **Eq. 5 and 6:** Detail the color and alpha formation for every planar node. Their form, utilizing differently parameterized and regularized neural fields (one purely position-dependent, the other both position- and view-dependent) and their unique interplay, is, to the best of our knowledge, unique to our method.\n\n- **Eq. 7:** Our flow field parameterization is also inspired by [6]; however, in our method, the flow is uniquely parameterized for every object and learned within its own frame of reference, whereas [6] applies optical flow in the camera coordinate system.\n\n**2. Missing Training Details**  \n\nThe reviewer noted that key training details are missing. Due to NeurIPS paper size limitations, we had to move all detailed training specifics into the  **Experimental Details (Sec. 2.) of our supplement**, as also noted within our main paper. We tried to cover all information within this dedicated section, and further **have uploaded all code in an anonymous github repository** (Sec. 4.1 Ln. 245) to circumvent any missing details.\n\n**3. Comparison with Atlas-based Methods**  \n\nThe reviewer remarked that the experiments were missing atlas-based approaches. We would like to clarify that  *we compare against established video editing and matting methods*, addressing this point directly. As reported in **Table 3 of our main paper**, NAGs demonstrate significant performance advantages (e.g., a 7.3 dB PSNR improvement) over these baselines, specifically **Layered Neural Atlases [3] (LNA)**—a recent texture editing method—and **OmnimatteRF [4] (ORF)**—a SOTA matting approach. This validates NAGs' robust decomposition and reconstruction capabilities even without relying on explicit 3D supervision, while simultaneously enabling flexible editing. The evaluation confirms that NAGs offer substantial benefits in terms of consistency, object decomposition, and general video editing over existing methods.\n\nTo further improve clarity, we will add a paragraph elaborating on these recent video matting methods and their specific usage.\n\n**Questions**\n\n**Q1**  \n\nFor the Waymo dataset, we rely on the camera poses provided directly by the dataset, consistent with the handling by our baselines. For the DAVIS dataset, we leverage RodynRF [7] for camera pose estimation, also consistent with our baseline OmnimatteRF [4]. Further comprehensive details regarding our camera modeling approach can also be found in the Method Details section (Section 1.1) in our supplement.\n\n**Q2**\n\nAll objects are used to initialize the system, as each object has its dedicated neural fields. We leverage the input masks to initially project the color and alpha information onto our base color and alpha estimates for each object. For every object (excluding the global background), we establish its initial 3D trajectory, which will be further optimized. For the Waymo dataset, this initialization utilizes the provided 3D bounding boxes, similar to [1]. For scenes where 3D bounding boxes are not available (e.g., DAVIS), object positions are estimated based on image-to-image homographies derived from sequential frames and an initial monocular depth estimation.\n\n**Q3**\n\nOur training strategy employs a **multi-phase optimization approach** combined with a **coarse-to-fine learning strategy** to effectively optimize the various components of each node. The training is subdivided into 3 phases over 80 epochs:\n\n- 1. (Epochs 0-5): Only the positional parameters ($\\mathcal{P}^{T}\\_i$,$\\mathcal{P}^R\\_i$,$\\mathcal{P}^{T}\\_{cam}$,$\\mathcal{P}^R\\_{cam}$) are optimized to correct for initial positional errors of objects and the camera.\n\n- 2. (Epochs 5-20): The color and opacity fields ( $\\mathcal{F}\\_{i, c}$,$\\mathcal{F}\\_{i, \\alpha}$) are additionally optimized, alongside the positional parameters.\n\n- 3. (Epochs 20+): All parameters ($\\mathcal{P}^{T}\\_i$,$\\mathcal{P}^R\\_i$,$\\mathcal{P}^{T}\\_{cam}$,$\\mathcal{P}^R\\_{cam}$,$\\mathcal{F}\\_{i, c}$,$\\mathcal{F}\\_{i, \\alpha}$,$\\mathcal{F}\\_{i, f}$,$\\mathcal{F}\\_{i, \\phi}$)  —are optimized together. Now additionally including flow, and view-dependent fields.\n\nIn conjunction with these phases, we apply a coarse-to-fine learning strategy by masking encoding dimensions for the view-dependent and planar flow fields, gradually activating more encoding dimensions as training progresses. This approach limits the initial expressiveness of the view-dependent field and encourages the planar flow field to learn coarse alignments first, before enabling finer details and view dependencies to emerge, thus guiding the overall optimization process. Further Details can be found in the supplementary in Sec. 1.2 and 2.2.\n\n**Q4** \n\nOur model distinguishes between **fixed initial conditions** and **learnable neural fields**:\n\n- **Fixed Initial Conditions:** $\\tilde{C}_i$ (base color) and $\\tilde{A}_i$ ​ (base alpha) are **not learnable** parameters of the network. They are initialized by an initial forward projection of a reference frame onto the position-initialized plane, based on the provided masks (Sec. 3.3 Ln 181f.). This step serves to pre-condition each atlas node to a dedicated region, providing a good starting point.\n\n- **Learnable Neural Fields**: The core appearance and motion are then captured by our *learnable* neural fields: the color field $\\mathcal{F}\\_{i, c}$​, opacity field $\\mathcal{F}\\_{i, \\alpha}$, flow field $\\mathcal{F}\\_{i, f}$, and view-dependent field $\\mathcal{F}\\_{i, \\phi}$​. These fields are explicitly designed and optimized to serve distinct roles:\n\n  - The color field $\\mathcal{F}\\_{i, c}$ and opacity field $\\mathcal{F}\\_{i, \\alpha}$ are primarily responsible for modeling the *view-agnostic, canonical appearance* of the object in its atlas space.\n\n  - The flow field $\\mathcal{F}\\_{i, f}$ warps this canonical appearance across time, enabling the representation of non-rigid motion and facilitating better editability by keeping the base texture consistent.\n\n  - The view-dependent field $\\mathcal{F}\\_{i, \\phi}$ is designed to capture subtle view-dependent effects (e.g., specularities, reflections) that cannot be explained by the canonical appearance or flow alone.\n\nThe optimization process distinguishes between these components through our phase-based learning and coarse-to-fine learning strategy. By initially limiting the expressiveness of the view-dependent field (via sparse encoding) and progressively activating it, we implicitly regularize the model to put the majority of the information into the non-view-dependent fields ($\\mathcal{F}\\_{i, c}, \\mathcal{F}\\_{i, \\alpha}, \\mathcal{F}\\_{i, f}$). This forces the learnable non-view-dependent components to capture the primary appearance and motion, which are then effectively mappable by the flow for enhanced editability. The view-dependent field then only incorporates additional, subtle changes, preventing conflation and ensuring a disentangled representation.\n\nWe will add these clarifications to the revised work.\n\n**References**\n\n[1] Chen, Ziyu, et al. \"OmniRe: Omni Urban Scene Reconstruction.\" The Thirteenth International Conference on Learning Representations. 2025  \n\n[2] Yang, Jiawei, et al. \"EmerNeRF: Emergent spatial-temporal scene decomposition via self-supervision.\" arXiv preprint arXiv:2311.02077 (2023).\n\n[3] Kasten, Yoni, et al. \"Layered neural atlases for consistent video editing.\" ACM Transactions on Graphics (TOG) 40.6 (2021): 1-12. \n\n[4] Lin, Geng, et al. \"Omnimatterf: Robust omnimatte with 3d background modeling.\" Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.\n\n[6] Chugunov et al. “Neural Spline Fields for Burst Image Fusion and Layer Separation”. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 25763–25773, 2024.\n\n[7] Liu et al. Robust dynamic radiance fields. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023.\n\n[17] Kopanas et al. “Neural Point Catacaustics for Novel-View Synthesis of Reflections”. ACM Trans. Graph., 41(6), 2022. Publisher: Association for Computing Machinery."}]

Author rebuttal:

Meta-review: The paper received three Accept and one Borderline Accept. All the reviewers appreciate that this paper proposes a novel object-centric representation for dynamic scene decomposition and editing, with a reasonable technical approach, good evaluation, and some interesting conclusions for the community. Thus, this paper achieves the NeurIPS acceptance requirement. However, AC highly urges the authors to go through the detailed comments carefully to polish the writing and provide extra information for comparison and completeness, so as to make sure the final acceptance.
Decision: Accept (spotlight)