EF-3DGS: Event-Aided Free-Trajectory 3D Gaussian Splatting
Authors: Bohao Liao
Wei Zhai
Zengyu Wan
Zhixin Cheng
Wenfei Yang
Yang Cao
Tianzhu Zhang
Zheng-Jun Zha
Venue: NeurIPS 2025 spotlight ()
Keywords: 3D Gaussian Splatting (3DGS); Novel View Synthesis (NVS)
Primary area: applications

Abstract: Scene reconstruction from casually captured videos has wide real-world applications. Despite recent progress, existing methods relying on traditional cameras tend to fail in high-speed scenarios due to insufficient observations and inaccurate pose estimation. Event cameras, inspired by biological vision, record pixel-wise intensity changes asynchronously with high temporal resolution and low latency, providing valuable scene and motion information in blind inter-frame intervals. In this paper, we introduce the event cameras to aid scene construction from a casually captured video for the first time, and propose Event-Aided Free-Trajectory 3DGS, called EF-3DGS, which seamlessly integrates the advantages of event cameras into 3DGS through three key components. First, we leverage the Event Generation Model (EGM) to fuse events and frames, enabling continuous supervision between discrete frames. Second, we extract motion information through Contrast Maximization (CMax) of warped events, which calibrates camera poses and provides gradient-domain constraints for 3DGS. Third, to address the absence of color information in events, we combine photometric bundle adjustment (PBA) with a Fixed-GS training strategy that separates structure and color optimization, effectively ensuring color consistency across different views. We evaluate our method on the public Tanks and Temples benchmark and a newly collected real-world dataset, RealEv-DAVIS. Our method achieves up to 3dB higher PSNR and 40% lower Absolute Trajectory Error (ATE) compared to state-of-the-art methods under challenging high-speed scenarios.

EF-3DGS: Event-Aided Free- Free-Trajyctory 3D Gaussian Splatting
Bohao Liao
Wei Zhai't
Zengyu Wan'
Zhixin Cheng' Zheng-Jun Zha?
Wenfei Yang*
Yang Cao
Tianzhu Zhang!
University of Science and Technology of China
(liaobh, wanzengy, chengzhixin]@mail.. ustc. edu. en, (wzhai056, yangwf, forrest, tzzhang,
edu. cn
discrete frames
continuous event stream
Free-traje tree-tajjettry video under high speed
COLMIITT
oo MAm
COLMMNTT
COLMANST
LocalRF
CF-3DGS
EvCF-3DGS
Figure 1: tree-trajectory 3DGS under high speed. (Top) The overall paradigm. The colored dots in the top row represent the event data (red: positive, blue: negative). We leverage continuous event streams to aid discrete video frames captured along free trajectories high-speed scenarios, jointly optimizing camera poses and reconstructing the 3DGS. Our method surpasses current state-of-the-art methods terms of both rendered results (middle) and pose estimation (bottom).
Abstract
Scene reconstruction from casually captured videos has wide real-world applica- tions. Despite recent progress, existing methods relying on traditional cameras tend fail in high-speed scenarios due insufficient observations and inacourate pose estimation. Event cameras, inspired by biological vision, record pixel-wise intensity changes asynchronously with high temporal resolution and low latency, providing valuable scene and motion information in blind inter-frame intervals. this paper, we introduce the event cameras to aid scene construction from a casually captured video for the first time, and propose Event-Aided Free- Trej-Tracctory 3DGS, called EF-3DGS, which seamlessly integrates the advantages of event cameras into 3DGS through three key components. First, we leverage the Event
"Fistt author I Corresponning author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Generation Model (EGM) to fuse events and frames, enabling continuous super- vision between discrete frames. Second, we extract motion information through Contrast Maximization (CMax) of warped events, which calibrates camera poses and provides gradient-domain constraints for 3DGS. Third, to address the absence of color information in events, we combine photometric bundle adjustment (PBA) with a Fixed-GS training strategy that separates structure and color optimization. effectively ensuring color consistency across different views. We evaluate our method on the public Tanks and Temples benchmark and a newly collected real- world dataset, RealEv-DAVIS. Our method achieves up to 3dB higher PSNR and 40% lower Absolute Trajectory Error (ATE) compared to state-of-ther tf--therrtttt art methods under challenging high-speed scenarios.
Introduction
In recent years, Neural Radiance Fields (NeRF) 1 2 3 and 3D Gaussian splatting (3DGS) 45 have made significant progress in novel view synthesis tasks. Given a set of posed images of the same scene, they optimize an implicit or explicit scene representation using volume rendering. While subsequent methods [2 3 6 excel with posed images, reconstructing scenes from videos with free camera trajectories remains challenging despite its applications in VR/AR, video stabilization, and mapping. To tackle this challenging task, several efforts have been made.
Accurate pose estimation is often difficult to obtain in free-trajectory scenarios, which directly impacts the quality of scene reconstruction. One line of work draws inspiration from Simultancous Localization and Mapping (SLAM). They g 5 9 follow its optimization paradigm, progressively optimizing camera trajectories and alternating between camera pose and scene refinement. Another explores incorporating additional geometric or motion priors such as depth estimation B3 or optical flow [15] establish constraints beyond photometric rendering
line of work [10]
99 I1
5 12
loss. While these methods can render photo-realistic images in typical free-trajectory scenarios, both their rendering quality and pose estimation accuracy degrade significantly high-speed scenarios (or equivalently low-frame-rate scenarios) as shown in Fig. [ Such high-speed scenarios have essential applications such as autonomous driving and First-Person View (FPV) exploration.
The performance degradation of prior methods can be attributed to two primary factors. First, the limited number of camera observations leads to an under-constrained scene reconstruction problem. This can cause the scene representation to converge to a trivial solution [16] 17 [2]. where the model overfits to the training views without capturing the correct underlying geometry structure. Second, the substantial discrepancies between consecutive frames, resulting in diminished overlapping regions, violate the implicit assumption of continuous motion between adjacent frames, which leveraged by previous methods. Moreover, geometric and motion priors like optical flow and feature matching become unreliable in such scenarios. These significant violations greatly exacerbate the ill ill-posedness of the joint optimization of scene and camera poses.
Event camera is a bio-inspired image sensor that asynchronously records per-pixel brightness changes, offering advantages such as high temporal resolution, high dynamic range, and no motion blur 21 22 23 The brightness information recorded in the event stream can effectively complement the missing scene information between consecutive frames. Moreover, the event data naturally encodes the motion information of the scene 24 25 26 containing rich motion cues. These properties make event cameras well-suited for scene reconstruction tasks in high-speed and free- trajectory scenarios. However, seamlessly integrating the aforementioned benefits of event cameras is nontrivial. First, 3DGS renders absolute pixel brightness, which aligns with image data. Event cameras, however, record sparse differential brightness changes. Directly integrating the differential operations into 3DGS may amplify noise and lead to ill-conditioned optimization problems with high sensitivity to parameter initialization and perturbations. Second, event cameras encode motion through continuous nettio-temporal trajectories of events. In contrast, frame-based data inherently discretizes continuous motion, forcing traditional methods to rely on correspondence matching, which fails in high-speed scenarios with large inter-frame displacements. These fundamental challenges require carefully designed method that bridges the gap between the event data and 3DGS optimization.
18
19
In this work, we propose Event-Aided Free- Trajectroy 3DGS, dubbed EF-3DGS, a framework that integrates event data into the scene optimization process to fully leverage its high temporal resolution property. Our approach comprises three key components: (1) In the Event Generation Model (EGM),
2

---

we introduce an event event-based re-render loss, which extends the 3DGS optimization to the continuous event stream. This allows us to utilize the brightness cues encoded in the event stream between adjacent frames, providing rich supervisory signals to alleviate the insufficient sparse view issues. (2) the Linear Event Generation Model (LFGMM, regarding the pose estimation challenge, we introduce the CMax [27 framework to exploit the spatio-temporal correlations of events. We obtain the motion field by leveraging the pseudo-depth from 3DGS rendering and the relative camera motion between consecutive frames. We then warp the events triggered by the same edge along the motion trajectories to maximize the sharpness of the image warped events (IWE), thereby estimating the motion that best matches the current spatio-temporal event patterns. Furthermore, through the LEGM [28 29]. we establish a connection between motion and brightness changes. This allows us to constrain the 3DGS in the gradient domain using the IWE. (3) As most event data primarily records scene brightness changes, lacking color information, we introduce photometric bundle adjustment (PBA) and Fixed-GS strategy to address this. PBA recovers color by optimizing reprojection errors onto RGB frames, while Fixed-GS enables separate optimization of scene structure and color.
Our main contributions are summarized as follows:
We introduce event cameras into the task of free-trajectory scene reconstruction for the first time. Its advantage of high temporal resolution and low latency showcases the potential of event data for scene reconstruction tasks in challenging scenarios.
We derive our method from the underlying imaging principles of event cameras and design the corresponding loss functions that mine the motion and brightness information encoded in event data and seamlessly integrate them into the 3DGS optimization.
Experiments on both public benchmarks and real-world datasets demonstrate that our method significantly outperforms existing state-of-the-art approaches in terms of both rendering quality and trajectory estimation accuracy.
2 Related Works
Joint Pose and Scene Optimization. The research community has recently focused developing 32 [S] that can be optimized without requiring precomputed camera poses. A line of work has focused on improving the stability of the optimization process. GARF 31 and BARF 32) both find that the high-frequency position encoding is prone to local minima and try to improve it. For example, GARF (31) proposes using Gaussian activation replace the sinusoidal position encoding. Another line of work has investigated incorporating additional constraints to make the problem more tractable. LocalRF [ leverages the prior assumption of continuous motion between adjacent frames and progressively adds and optimizes camera poses. More recent approaches 10, & 5 leverage pre-trained networks, Le., monocular depth estimation and optical flow estimation. Exploiting 3DGS's s explicit representation, CF-3DGS [5] directly back- projects Gaussian points using depth maps. While the aforementioned methods have made notable progress, they have yet to fully address the challenges posed by high-speed scenarios rely good pose initialization. Our approach addresses these issues by leveraging motion and brightness cues from event streams.
methods [12 g 30 31
10
Event- Based Novel View Synthesis. Recent works have explored the integration of event cam- eras 33 34 351 into the NeRF or 3DGS framework. Early approaches, such as E-NeRF (36 and EventNeRF B7]. utilize event-based generative models, minimizing the difference between the ren- dered brightness changes and observed brightness changes. Building upon this, Robust e-NeRF [38 incorporates a more realistic imaging model into the event-based framework, accounting for factors like refractory periods and noise. Beyond event-based NeRF, efforts have also been made to integrate event data into image- methods. For instance, E2NeRF [39] and EvDeblurNeRF [40] leverage the Event Double Integral (EDI) [4]] model to address the deblurring problem, while DE-NeRF 42 and EvDNeRF 43 leverage the high temporal resolution property event cameras to capture fast-moving elements in dynamic scene. More recently, Event-3DGS 44 and EaDeblur-Gt [45 have extended previous approaches to 3D Gaussian Splatting, achieving superior rendering quality and real-time performance. A key distinction of our work is that, unlike the prior methods that rely on accurate precomputed poses, we target free-trajectory scenarios, jointly optimizing for both the camera poses and the scene representation. Furthermore, while previous works have been limited to simulated and simple environments, we evaluate our approach in large-scale outdoor scenarios with complex motions and lighting conditions.
3

---

Input
event stream
latent intensity
rendered
subinterval event image
(1.1) Event Generation Model
Stage Event Driven Optimization
color distotionn
gaussian
attc:
now worping
progressively add
warped event imagee
Linear Event Generation Model
Stage
Training
color
LySA
optivito color only
THED
reprojected
Insgge
(1.3) Photometric Bundle Adjustment
a. Gaussin
parann
optiriizd garam
Figure 2: Method overview. The inputs are video frames and event stream. In the first stage, we progressively add new event images, leveraging the events and most recent frame to establish the event-driven optimization. In the second stage, we adopt the Fixed-GS strategy to mitigate the color distortion of 3DGS. The details of LIEgMM and CMax framework are shown in Fig.
Preliminary
3DGS 4 parametrizes the 3D scene set of 3D gaussians {G+] that carry the geometric and appearance information. Each 3D Gaussian is characterized by several Iearnable properties, including its center position μ E IR3 opacity E [0. 1], spherical harmonics (SH) features fą 3x for view-dependent color E IR", rotation matrix R E R3x3 (stored in quaternion form), scale factor E 3 The shape of each Gaussian defined by the covariance matrix Σ and the center (mean) point μ4, G(z) exp(-1 2 (a- -1) " ""'>i(4 1x p)). During rendering, a bile-based rasterizer is applied to enable fast sorting and a-blending. The color of each pixel is calculated via blending N ordered overlapping points:
N
C(r) =
Σ CO II - (1)))), 1=1
where is calculated from spherical harmonics and view direction, a, is the multiplication of opacity and the transformed 2D Gaussian and r denotes the image pixel. With the forward rendering procedure,
optimize 3DGS by minimizing a weighted combination loss of L1 and LD- SSIM (f, + XLD-SSIM( T), where By integrating depth d, in Equation (T) along
between observation and rendered pixels: Lestor
balancing weight which is set to 0.2 following 4
the ray, we can also obtain a expected depth value D(r):
D(r) Σ daa; (1 a;))
f=1
Method
The overall framework is shown in Fig. 2 Given video of free-trajectory {I+ captured
time {t,} and the event stream E = {cx]- our goal is to reconstruct the 3DGS of the scene and the corresponding camera trajectory Following the maalssi-bb--yynteee
paradigm of 3DGS, extend this approach by incorporating event camera data through two fundamental imaging principles: the Event Generation Model (EGM) and Linear Event Generation Model (LEGM).. To address the absence of color information in events and ensure cross-view consistency, we further introduce photometric bundle adjustment (PBA) and Fixed-GS training strategy.

---

gndient flww
ST
lineorized EOM
optical Rlow
(I)))
||PWE St.
PWEIAL)
feef
Figure 3: The illustration of unified CMax and LEGM optimization. We warp previous event frames to the sampled timestamp through the optical flow and maximize the sharpness of the image of IPWE. The byproduct IPWE is utilized to establish additional constraints 3DGS.
4.1 EGM Driven Optimization
The EGM describes how event cameras asynchronously record pixel-wise brightness changes. When the logarithmic brightness change at a pixel uy (Ik. 1/a), exceeds a predefined contrast threshold C,
AL tx) L( (x) L (u, 1 t 8t) = C&C,
where I. log(7) is the logarithm of intensity, Pk: (-1, +1} indicates the polarity of brightness changes, and f the triggered timestamp.
As shown in Fig. (1.1), ), to leverage the high temporal resolution of events, we first divide the time interval between adjacent video frames I; and 4441 into N smaller subintervals εi = {eew [4: S I S 1.--1: 1: At higher temporal resolution:
--+1-4 N 14g = + At). This allows us form accumulated event frames at
Ei,
Σ Ps.
ELERL
We then reconstruct the latent intensity image I, at any intermediate time E (tr,) by integrating the accumulated events with the most recent frame:
4.00 14,0
exp(E Bin C) if >
I, I
(5)
if =
This latent intensity image provides a supervisory signal for event-based rendering loss:
LEGM = (1-
(6)
By enforcing consistency between rendered and latent intensity images, this loss effectively utilizes the brightness information encoded in event streams between adjacent frames, addressing the challenge of sparse viewpoints in high-speed scenarios.
4.2 Unified CMax and LEGM Optimization
While LEGM leverages the brightness change information recorded by events, it does not explicitly exploit the motion information encoded in the event stream. To address this, we introduce the
Contrast Maximization (CMax) 27 46 47 (88 framework and the LEGM [28 29
These
models complement the previous EGM-driven optimization.
Under constant scene illumination, events are triggered by the motion of scene edges, forming continuous trajectories in (x, y, t) space. As shown in Fig. 2ig.2 (1.2), by (arping(backkprojeee events along the correct motion trajectories, we can obtain sharp image of warped events (IWE). Therefore, the sharpness of the IWE can serve as an indication of the accuracy of the estimated motion. This insight motivates us to derive the motion field by leveraging the rendered depth from 3DGS using eq. and the relative camera motion between neighboring timestamps. By optimizing the sharpness of the IWE, we can obtain the optimal motion field, which turn helps to improve the geometric accuracy of the 3DGS and the camera poses.

---

As shown in Fig. 3 for efficiency, adopt wise warping approach instead of warping individual events. Specifically, for current timestamp Eyef ti we warp the event frames from previous r sub-intervals:
E j-- warp( Pig--jmm
where r0, FL.
the optical flow derived from the rendered depth 1 in eq. ( and relative pose -m between two timestamps:
m-+j Pij-m++j
Paj-j-m ( II (r, (2,, v),
(8)
where Ti.j---m = T.j--m T.j IT projects a 3D point to image coordinates and IT-1 unprojects pixel coordinate and depth into 3D point. Then the image of piece-wise warped events (IPWE) at timestamp fi is computed by averaging the warped event frames:
IPWE.
Eim- CAL. m=j-r
Σ
Following the Cmax framework, we maximize the variance of the IPWE, which is equivalent minimize its opposite:
Lem Var(W
(10)
Furthermore, based on the LEGM by the dot product of the image gradient VL and the optical flow ú (note that L is the logarithm of an image):
the brightness change AL pixel can be approximated
AT( VL = I((u) L(u + úi.
(11)
It is noteworthy that the IPWE also encodes brightness change information. Combining eq. ( and eq. [1]. we establish a connection between the IPWE and the brightness changes of the rendered images:
C IPWE,, = i(u) - L(u + Fig--+4).
(12)
Note that to compute Fijj-j+1. we estimate Ti.++ by leveraging the assumption of locally linear motion from Ti,--1 and T.g. Based on this relationship, we formulate an additional gradient-based loss:
Cgrad |IC IPWE,, (i(u)= -(u++ + Fig-)4+1)l1
(13)
where i. is the logarithm of synthesised image i, Finally, the full LEGM loss defined as:
LLEGM Aem + >Agradh
(14)
where Acm and Agrad are the balancing weight.
4.3 Photometric Bundle Adjustment
The aforementioned event- based constraints, LEGM and CLEGM, leverage the brightness change and motion information encoded the event data constrain 3DGS. However, as event cameras only record brightness changes and lack color perception, directly applying them to 3DGS optimization may lead inconsistent color rendering. To ensure cross-vier view consistency of the 3DGS rendering, we introduce the Photometric Bundle Adjustment (PBA) term.
Specifically, as shown in Fig. 2 (1.3), for randomly sampled timestamp E {4is], we establish the following photometric reprojection error:
LPBA = Σ Σ UEP J.EJ
[|4(4') i(u))l2
(15)
where u' I[T..,--=))IT" the pixel of source view T., P denotes the pixel samples of current frame, and F is the candidates of target video frames. We select F to be the nearest previous video frame consideration of computation costs.
y, D(u))) represent the coordinate on target view projected from

---

Micheds
Pase-Free
Input
6.855
12105.
PSNNE
EITRS
139051
PSNRRT
129055
PSNNE
5.EPS
1790S
PISRRT
TPURST
13.86
15.81
0.67
13.75
00.07
13.90
0.51
11.00
13.33
08:51
C.A
LacalRnn
g7
73344
0.35
0.39
23.00
0.71
0.70
33.48
11.99
03:00
71 mm
0.65
08:44
19:41
0163
6-41
76.05
0.78
0.51
0.35
73.75
0.39
27.06
0.42
70.33
0.05
C-M
ByDeHluNoRF
ExP
31.25
0:42
25.09
0.09
17.50
0.55
15:19
6.90
ENGRE
Exp
2362
0.33
0.37
22.84
0.70
0.38
21:85
0.00
0.41
20.50
0.65
0.06
18.09
0.00
0:53
ExF
36.33
0.78
0.33
0.76
0.36
34.99
0.37
64
0.72
0:38
22.41
0.00
0.38
E48
28,66
0.79
36,01
9.79
0.30
25.38
0.31
34.43
0.34
23.9
0.72
0.36
Table 1: Quantitative evaluations Tanks and Temples dataset. The best results are highlighted in bold. Note that, in the "input" column, "F" denotes traditional frame input, while "E+F" denotes hybrid frame and event input.
By minimizing LA across sampled views, we encourage the 3DGS model produce geometrically and photometrically consistent renderings across events and video frames, thus effectively resolving color inconsistencies inherent in event data.
4.4 Fixed-GS Training Strategy
The LABa term alone is insufficient to fully mitigate color distortion issues. To further address this challenge, we propose two-stage Fixed-GS scene optimization strategy that takes advantage of 3DGS's explicit attribute representation. In the first stage, all the parameters are optimizable and the optimization is performed across all timestamps:
Gi Tij
argmin Levents t E {tish,
(16)
where 1e, ar r, 8, is the position, opacity, rotation, scale factor and spherical harmonics of the Gaussians, and is the sampled timestamp during training. This stage results a scene reconstruction with accurate structure and brightness, albeit with potential color distortions due to the dominant colorless event supervision overwhelming the sparse RGB frame color supervision. The second stage focuses on recovering accurate color information. During this phase, optimization conducted exclusively on video frames. We optimize only the spherical harmonic coefficients of the Gaussians while keeping other parameters fixed:
G7 argmin Leotor, E (tr.u}
(17)
The ratio between the first and second stages is empirically set to 4:1. This approach allows us to effectively address the color distortion problem while preserving the structural and brightness information obtained from the event data.
4.5 Overall Training Pipeline
Assembling all loss terms, we get the overall loss function:
Levent LEGM LLEGM XPBACPBA.
(18)
where λPвA are the weighting factor. Note that since event cameras typically record only the changes brightness intensity, the LEGAS and LLEGM losses are computed in the grayscale domain, whereas the LPBA loss is calculated in RGB color space. We incorporate dynamic scene allocation strategies from LocalRF for handling extended video sequences. Our overall training pipeline builds upon the progressive optimization scheme of CF-3DGS [5] while introducing novel components to integrate event stream data for robust free-trajectory scene reconstruction. Please refer to Section A.3 for the algorithm pipeline and additional implementation details.
Experiments
5.1 Dataset
Tanks and Temples. We conduct comprehensive experiments the Tanks and Temples dataset [00 Similar to LocalRF [8) we adopt scenes, covering large-scale indoor and outdoor scenes. For each scene, we sample video clip with 50-second duration, typically featuring free camera trajectories
7

---

Methods
Ingust
RPF,
RPF
6.988
ATE.
RP6.
RPE.
ATE.
RFF.
1PPS
ATE
MPR.L
RPE,
PPP
ATF.
RFPE
RFF
1 FPS
ATEC
NepeNGRF
01141
117553
18982
10542
2.7653
0.220
1.3634
2.757
0:3151
13965
235-12
06216
3911
28392
LocaRF
0.080
0.9283
0.530
0.867
0.9885
0.6055
0.911
0.800
0.6301
0.0057
10428
0.800
0.421
14725
180066
CF-3DGS
0,0594
06681
0.412
0.637
0.7128
0.463
00712
0.733
0.5189
0.0899
0.804
0,0118
0 1057
0.976
038972
B:CT-JDGGS
Enf
0.8661
0,5972
0.5119
0.090
0.6289
0.3766
08538
0.622
0.4261
0.8891
0.7054
0.486
0.057
0.7597
0.5534
EP 3DUSIOurS)
Exp
00091
0,5427
0.2885
0.0407
8.521
0.3064
004466
0.579
0.337
0,0489
0.5953
0.3671
0.887
0,8259
0.3753
Table Pose accuracy on Tanks and Temples. We use COLMAP poses in Tanks and Temples as the "ground truth". The unit of RPE, is in degrees, ATE is in the ground truth scale and RPE, is scaled by 100. Those methods that require precomputed poses are excluded.
CF-3DGS
wvent33DGSEE+F)
EvCF-3DGS
EF-3DGS(OOrs)
Gruth
Qualitative comparison for novel view synthesis. The first two rows are from Tanks and Temples and the last row from RealEv-DAVIS. Our approach produces more realistic rendering results with fine-graines details. Better viewed when zoomed in.
Figure 4:
and covering considerable distance. Following LocalRF [1, apply 4x spatial downsampling to the videos. To evaluate the robustness under varying camera speeds, we employ varying temporal downsampling of 6 FPS, 4 FPS, 3 FPS, 2 FPS, and 1 FPS. The reduction in frame rate effectively creates larger inter- frame displacements, simulating high-speed scenarios. To synthesize realistic event data, we first upsample the original videos by [5]] and then apply the simulator V2E (52).
RealEy-DAVIS. Due to the lack of free-trajectory event camera datasets, we introduce RealEv- DAVIS. Using DAVIS346 camera that simultaneously captures frames and events at 346x260 resolution, we record 40-second handheld sequences at 25 FPS. We employ COLMAP for ground- truth poses. For SLOW scenarios, retain every second frame. while for FAST scenarios, keep only one frame per five frames. Further details are provided in Section A.2
5.2 Implementation details
We follow the optimization parameters by the configuration outlined in the 3DGS [4] We optimize the camera poses in the representation of quaternion rotation. The initial learning rate is set to 10 and gradually decays to 10 6 until convergence. The balancing weight Aems Agrad and XPBA is empirically set 0.1, 0.2 and 0.5. For the division of events between adjacent frames,
maintain
a constant interval of 1s for Tanks and Temples and 1 25 s for RealEv-DAVIS, setting the number of subinterval N accordingly. For example, in Tanks and Temples, N equals 2 for 3FPS and 6 for IFPS. This ensures adherence to the constant brightness assumption within each sub-interval and provides adequate events for the following CMax warping. The intervals of neighboring warping r in CMax are set to 3. The contrast threshold C' set to 0.25 for Tanks and Temples and 0.21 for RealEv-DAVIS. We provide detailed ablation studies on these hyperparameters and additional implementation details in Section A.4
5.3 Experimental Setup
Metrics. We evaluate all the methods from two aspects: novel view synthesis and pose estimation. For the novel view synthesis task, we report the standard metrics PSNR, SSIM [53], and LPIPS [54]. For the pose estimation task, we adopt the Absolute Trajectory Error (ATE) and Relative Pose Error
(RPE) metrics (55 56), delineated in [10. Since these metrics are inherently influenced by frame rate, we upsample estimated poses consistent temporal resolution before evaluation for fair comparison across different frame rate settings.
8

---

Temples Panther
Loody
FOr 2014
144
Powns
Frew
Fonta
D20 2004
Can
RealEv DAVVIS Outdoor
-22
Flare
Froms
Frew D
Forte
Figure 5: Pose estimation comparison. We visualise the trajectory (3D plot) and RPE, (color bar) of each method. We clip and normalize the RPE, by a quarter of the max RPE, across all results of each scene.
SLOW
FAST
Methods
Input
NVS
Pose
NVS
Pose
PSNR1
SSIM
RPE,
RPE,
PSNR
SSIM
RPE.
RPE-1
LocalRF CF-3DGS
20,83
0.6074
3.60
2.07
17.62
0.5192
22.68
0.6287
2.49
1.55
17.59
0.5204
3.68
2.17
EvDeblurNcRF
E+F
20.61
0.6064
17.98
0.5269
Event-3DGS (E+F)
R+F
23.43
0.6456
20.04
0.5515
EvCF-3DGS
R+F
22.89
0.6317
1.78
0.82
19.13
0.5380
2.70
1.28
BF-3DGS(Ours)
B+F
23.65
0.6466
1.41
0.69
21.12
0.5620
1.80
0.89
Table 3:
Rendering and pose estimation results on RealEv-DAVIS. Complete data and additional metrics are provided in the supplementary material.
Baselines. For fair comparison, we focus two categories of methods: (1) For frame-based
approaches, we selected methods specifically addressing free-trajectory scenarios, such as LocalRF and F2-NeRF [57]. We also include pose-free methods like Nope-NeRF [10] and CF-3DGS (5) (2) For event-frame hybrid methods, we consider approaches that fuse events and frames, including ENeRF (36,, EvDeblurNeRI [40] and Event-3DGS 44] Since no existing method integrates events for free-trajectory scenarios, we implement EvCF-3DGS as a competitive baseline that leverages an event-based frame interpolation network (Time Lens (58) temporally upsample frames before feeding them into CF-3DGS.
5.4 Experimental Results
We select every ten frames as test image for NVS evaluation following LocalRF g camera poses are unknown in our setting, we need to estimate the poses of test views. As in iNeRF [59] freeze the 3DGS model, initialize the test poses with the poses of the nearest training frames, and optimize the test poses by minimizing the photometric error between rendered images and test views.
Since the
Results on RealEv-DAVEL Table validates our approach on the real-world RealEv-DAVi dataset. EF-3DGS outperforms perforrrrrmnn methods and handles real-world scenes effectively. In FAST scenarios, our method shows nearly 1dB PSNR improvement over the best baselines. This confirms advantage in high-speed scenarios where frame-based methods struggle. Fig, 4 and Fig. F our method preserves fine details and maintains accurate trajectories even during rapid motion, addressing key limitations of traditional approaches.
Escu Ciocu Lons Fixe GS
PENRT
SSIM
NVS
LPPPS
RPE, RPE J ATE
Posc
20.53
0.55
0.95
0.1057
0.9765
0.8972
22.15
0.68
0.42
0.0651
0.7529
0.5770
21.07
0.67
0.54
010830
0.5869 0.7231
20.95
0.05
0.35
0.0938
0.9875 0:9112
22.85
0.68
0.90
0.0523
0.6387 0.398]
CCECUAL
2- 2-D0BCL
23.45
0.70
0.37
0.0623
0.6387 0.398]
23.09
0.70
0.38
0.6487
0,6299 0.3783
23.96
0.72
0.36
0.0487
0.6259 0.3753
Figure 6: Robustness of different meth- ods to pose disturbance.
Table 4: Effect of each component in EF-3DGS. The best
results are highlighted in bold.

---

Results on Tanks and Temples. Tables I and demonstrate two key findings: (1) Our event-aided approach achieves up to 3dB higher PSNR and nearly 40% lower trajectory error at IFPS compared to frame-based methods, indicating the critical value of event data in high-speed scenarios. (2) Our method maintains 1.55dB PSNR advantage over Event-3DGS at IFPS, confirming that our integration framework effectively exploits the nature of event data beyond merely using it. Fig. 4 shows our method produces sharper edges and finer textures, while Fig. 5 iillustrates we achieve more accurate trajectory estimation.
Performance under Varying Camera Speeds As shown in Table 2 and Table 3 while all methods degrade as the frame rate decreases, our approach shows remarkable resilience. The performance gap lower frame rates, with our PSNR advantage over CF-3DGS (5) increasing from 0.6 ddB at 6FPS to 3. 3.43dB at IFPS. Notably, our method also consistently outperforms other event-based methods DEvCF-3DGS and Event-3DGS) This confirms not only the value of event data in challenging scenarios but also the superiority of our integration approach.
widens significantly
5.5 Ablation Studies
Effect of Each Component Table 4 presents comprehensive ablation study of our key components under the challenging IFPS setting on Tanks and Temples. LEGM serves the foundation of our approach, providing substantial improvements in both rendering quality (+1 ++1.63dB PSNR) and pose accuracy by enabling rich supervision between discrete frames. Building upon this, LLEGM extracts motion information from events and constrains 3DGS in the gradient domain, significantly improving pose estimation while modestly enhancing rendering quality. LPBA, though designed to address color inconsistency issues, not only improves rendering quality but also enhances pose estimation accuracy by establishing geometric and photometric consistency across views. The Fixed-GS training strategy, while having no impact pose optimization, significantly improves rendering quality by effectively separating structure and color optimization. We provide more intuitive ablation visualizations in Section A.5
Robustness to Pose Disturbance To validate the robustness of different methods under inaccurate pose initialization, a common challenge in practical scenarios, we introduce varying degrees of perturbations to the initial camera poses estimated by COLMAP. Specifically, following BARF [2) we parametrize the camera poses p with the se(3) Lie algebra. For each scene, we synthetically perturb the camera poses with additive noise óp ~ NO, nll, where is the noise level. Then, each method is initialized with the noised poses, after which the optimization performed. The results are illustrated in Fig. 6 Notably, Event-3DGS [4), which lacks the capability to optimize camera poses, exhibits a drastic performance degradation as the magnitude of pose disturbances increases. This observation validates the critical importance of joint pose-scene optimization. Furthermore, Our proposed framework demonstrates superior tolerance across all perturbation levels. Even under significant noise, our method experiences substantially less degradation in both rendering quality and trajectory accuracy.
Conclusions
In this work, we propose Event-Aided Free- crej-Trajyyyoy 3DGS (EF-3DGS), a novel framework that seamlessly integrates event camera data into the task of reconstructing 3DGS from casually captured free-trajectory videos. Our method effectively leverages the high temporal resolution and motion information encoded in event streams to enhance the 3DGS optimization process, leading to improved rendering quality and accurate camera pose estimation. By introducing the Event Generation Model and Linear Event Generation Model, we bridge the gap between differential event data and absolute brightness rendering in 3DGS. The proposed photometric bundle adjustment and Fixed-GS strategy further ensure accurate color recovery and scene structure optimization. Extensive experiments on both public benchmarks and real-world datasets validate the effectiveness of our approach, demonstrating significant improvements over sate-of-the-as art methods in both rendering quality and trajectory estimation accuracy. Future work would explore addptivee parameter adjustment strategies enhance the method's 's versatility and ease of use across various reconstruction tasks.
7 Acknowledgements
This work is supported by the National Natural Science Foundation of China (NSFC) under Grants 62225207, 62436008 and 62306295. The AI-driven experiments, simulations and model training were performed on the robotic Al-Scientist platform of Chinese Academy of Sciences.
10

---

References
[1] Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. In ECCV, 2020.
[2] Jonathan T. Barron, Ben Mildenhall, Dor Verbin, Pratul P. Srinivasan, and Peter Hedman. Mip-nerf 360: Unbounded anti-aliased neural radiance fields. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5470-5479, June 2022.
[3] Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphics primitives with a multiresolution hash encoding. ACM Trans. Graph., 41(4), jul 2022.
[4] Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendcring. ACM Transactions on Graphics, 42(4), July 2023.
[5] Yang Fu, Sifei Liu, Amey Kulkarni. Jan Kautz, Alexei A. Efros, and Xiaolong Wang. Colmap-free 3d gaussian splatting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 20796-20805, June 2024.
[6] Anpei Chen, Zexiang Xu, Andreas Geiger, Jingyi Yu, and Hao Su. Tensorf: Tensorial radiance fields. In Shai Avidan, Gabriel Brostow, Moustapha Cissé, Giovanni Maria Farinella, and Tal Hassner, editors, Computer Vision
ECCV 2022. pages 533-350 Cham, 2022. Springer Nature Switzerland.
[7] Jonathan Barron, Ben Mildenhall. Dor Verbin, Pratul P. Srinivasan. and Peter Hedman. Zip-nerf: Anti-aliased grid-based neural radiance fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 19697-19705, October 2023.
[8] Andréas Meuleman, Yu-Lum I Liu, Chen Gao, Jia-Bin Huang, Changil Kim, Min Kim, and Johannes Kopf. Progressively optimized local radiance fields for robust view synthesis. In Proceedings of the IEEF/CVF Conference Computer Vision and Pottern Recognition (CVPR), pages 16539-16548, Junc 2023.
[9] Kaiwen Jiang, Yang Fu, Mukund Varma T, Yash Belhe, Xiaolong Wang, Hao Su, and Ravi Ramamoorthi. A construct-optmmiee approach to sparse view synthesis without camera pose. In ACM SIGGRAPH 2024 Conference Pupers, pages 1-11, 2024.
[10] Wenjing Bian, Zirui Wang, Kejie Li, Jia- Ha-Wang Bian, and Victor Adrian Prisacariu. nope-nerf: Optimising neural radiance field with no pose prior. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4160-4169, June 2023.
[11] Yu-Lun Liu, Chen Gao, Andreas Meuleman, Hung- Yu Tseng, Ayush Saraf, Changil Kim, Yung-Yu Yu Chuang, Johannes Kopf, and Jia-Bin Huang. Robust dynamic radiance fields. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13-23, 2023.
[12] Prune Truong. Marie-Julie Rakotosaona. Fabian Manhardt. and Federico Tombari. Sparf: Neural radiance fields from sparse and noisy poses. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4 4190-4200, June 2023.
[13] S. Mahdi H. Miangoleh, Sebastian Dille, Long Mai, Sylvain Paris, and Yagiz Aksoy. Boosting monocular depth estimation models to high-resolution via content-adaptive multi-resolution merging. In Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 9685-9694, June 2021.
[14] René Ranftl, Alexey Bochkovskiy, and Vladlen Koltun. Vision transformers for dense prediction. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 12179-12188, October 2021.
[15] Zachary Teed and Jia Deng. Raft: Recurrent all-pairs field transforms for optical flow. In Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michacl Frahm, editors, Computer Vision ECCV 2020, pages 402 419, Cham, 2020. Springer International Publishing.
[16] Michael Niemeyer, Jonathan T. Barron, Ben Mildenhall, Mehdi S. M. Sajjadi, Andreas Geiger, and Noha Radwan. Regnerf: Regularizing neural radiance fields for view synthesis from sparse inputs. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5480-5490, June 2022.
[17] Guangcong Wang, Zhaoxi Chen, Chen Change Loy, and Ziwei Liu. Sparsenerf: Distilling depth ranking for few-shot novel view synthesis. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 6076, October 2023,

---

[18] Guillermo Gallego, Tobi Delbrück, Garrick Orchard, Chiara Bartolozzi, Brian Taba, Andrea Censi, Stefan Leutenegger, Andrew J. Davison, Jörg Conradt, Kostas Daniliidis, and Davide Scaramuzza. Event-based vision: survey, IEEE Transactions on Puttern Analysis and Muchine Intelligence, 44(1):154-188) 2022.
[19] Javier Hidalgo-Carrió, Guillermo Gallego, and Davide Scaramuzza. Event-aided direct sparse odometry. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5781-5790, June 2022.
[20] Antoni Rosinol Vidal, Henri Rebecq. Timo Horstschaefer, and Davide Scaramuzza. Ultimate slam? combining events, images, and imu for robust visual slam in hdr and high-speed scenarios. IEEE Robotics and Automation Letters, (00, 2018.
[21] Zengyu Wan, Yang Wang. Zhai Wei, Ganchao Tan, Yang Cao, and Zheng-Jun Zha. Event-hased optical flow via transforming into motion-dependent view. IEEÉ Transactions on Image Processing, 2024.
[22] Yuliang Wu, Ganchao Tan, Jinze Chen, Wei Zhai, Yang Cao, and Zheng-Jun Zha. Event-based asyn- chronous hdr imaging by temporal incident light modulation. Optics Express, 32(11):1857-118388 2024.
[23] Chengzhi Cao, Xueyang Fu, Yurui Zhu, Zhijing Sun, and Zue Zha. Event-driven video restoration with mniking-convolutiona architecture. IEEE Transactions on Neural Networks and Learning Systems, pages 1- 15, 2023.
[24] Daqi Liu, Alvaro Parra, and Tat-Jun Chin. Globally optimal contrast maximisation for event-based motion estimation. In Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recognition (CVPR), June 2020.
[25] Ganchao Tan, Zengyu Wan, Yang Wang, Yang Cao, and Zheng-Jun Zha. Tackling event-based lip-reading by exploring multigrained spatiotemporal clues. IEEE Transactions on Neural Networks and Learning Systems, pages 1-13, 2024.
[26] Andrés Ussa, Chockalingam Senthil Rajen, Tarun Pulluri, Deepak Singla, Jyotibdha Acharya, Gideon Fu Chuanrong, Arindam Basu, and Bharath Ramesh. A hybrid neuromorphic object tracking and classifi- cation framework for real-time systems. IEEE Transactions on Neural Networks and Learning Systems, 15(8):10726-11735 2024.
[27] Guillermo Gallego, Henri Rehecq, and Davide Scaramuzza. unifying contrast maximization framework for event cameras, with applications to motion, depth, and optical flow estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018.
[28] Guillermo Gallego, Christian Forster, Elias Mueggler, and Davide Scaramuzza. Event-based camera pose tracking using a generative event model, 2015.
[29] Daniel Gehrig, Henri Rebecq, Guillermo Gallego, and Davide Scaramuzza. Asynchronous, photometric feature tracking using events and frames. In Proceedings of the European Conference on Computer Vision (ECCV). September 2018.
[30] Zirui Wang, Shangzhe Wu, Weidi Xie, Min Chen, and Victor Adrian Prisacariu. Nerf-: Neural radiance fields without known camera parameters, 2022.
[31] Shin-Fang Chng. Sameera Ramasinghe, Jamie Sherrah, and Simon Lucey. Gaussian activated neural radiance fields for high fidelity reconstruction and pose estimation. In Shai Avidan, Gabriel Brostow, Moustapha Cissé, Giovanni Maria Farinella, and Tal Hassner, editors, Computer Vision ECCV 2022, pages 264-280, Cham, 2022. Springer Nature Switzerland.
[32] Chen-Hsuan Lin, Wei-Chiu Ma, Antonio Torralba, and Simon Lucey. Barf: Bundle-adjusting neural radiance fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 5741-5751, October 2021.
[33] Zengyu Wan, Wei Zhai, Yang Cao, and Zhengjun 7ha. Emotive: Event-guided trajectory modeling for 3d motion estimation. arXiv preprint Xivv 11371, 2025.
[34] Han Han, Wei Zhai, Yang Cao, Bin Li, and Zheng-jur jun Zha. Event- tracking any point with motion- augmented temporal consistency. arXiv preprint arXiv:2412.013300 2024.
[35] Yifeng Cheng, Alois Knoll, and Hu Cao. Urnet: uncertainty- aware refinement network for event- event-based stereo depth estimation. Visual Intelligence, 3(1):18, 2025.
12

---

[36] Simon Klenk, Lukas Koestler, Davide Scaramuzza, and Daniel Cremers. E-nerf: Neural radiance fields from moving event camera. IEEE Robotics and Automation Letters, 83))11887-1944 2023.
[37] Viktor Rudnev, Mohamed Elgharib, Christian Theobalt, and Vladislav Golyanik. Eventnerf: Neural radiance fields from single colour event camera. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4992-5002, June 2023.
[38] Weng Fei Low and Gim Hee Lee. Robust e-nerf: Nerf from sparse & noisy events under non-uniform motion. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 18335-18346, October 2023.
[39] Yunshan Qi, Lin Zhu, Yu Zhang, and Jia Li. E2nerf: Event enhanced neural radiance fields from blurry images. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 13254-13264, October 2023.
[40] Marco Cannici and Davide Scaramuzza. Mitigating motion blur in neural radiance fields with events and frames. In Proceedings of the JEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9286-9296, 2024.
[41] Liyuan Pan, Cedric Scheerlinck, Xin Yu, Richard Hartley, Miaomiao Liu, and Yuchao Dai. Bringing a blurry frame alive at high frame-rate with an event camera. In Proceedings of the IEEF/CVF Conferencce Computer Vision and Pattern Recognition (CVPR), June 2019.
[42] Qi Ma, Danda Pani Paudel, Ajad Chhatkuli, and Luc Van Gool. Deformable neural radiance fields using rgb and event cameras. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 3590-3600, October 2023.
[43] Anish Bhattacharya, Ratnesh Madaan, Fernando Cladera, Sai Vemprala, Rogerio Bonatti, Kostas Danilidis. Ashish Kapoor, Vijay Kumar, Nikolai Matni, and Jayesh K. Gupta. Evdnerf: Reconstructing event data with dynamic neural radiance fields. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 5846-5855, Jamuary 2024.
[44] Haiqian Han, Jianing Li, Henglu Wei, and Xiangyang Ji. Event-3dgs: Event-based 3d reconstruction using
3d gaussian splatting. Advances in Neural Information Processing Systems, 37:128139 18811199911155 2025.
[45] Yuchen Weng, Zhengwen Shen, Ruofan Chen, Qi Wang, and Jun Wang. Eadeblur-gs: Event assisted 3d deblur reconstruction with gaussian splatting. arXiv preprint arXiv:2407.13520 2024.
[46] Xin Peng, Ling Gao, Yifu Wang, and Laurent Kneip. Globally-optimal contrast maximisation for event cameras. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(7):3479-3495 2022.
[47] Timo Stoffregen and Lindsay Kleeman. Event cameras, contrast maximization and reward functions: An analysis. Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), 2019.
[48] Shuang Guo and Guillermo Gallego. Cmax-slan Event-based rotational-motian bundle adjustment and slam system using contrast maximization. IEEE Transactions on Robotics, 2024.
[49] Zelin Zhang, Anthony Yezzi, and Guillermo Gallego. Formulating event-based image reconstruction linear inverse problem with deep regularization using optical flow. IEEE Transactions on Puttern Analysis and Machine Intelligence, page 1-18, 2022.
[50] Arno Knapitsch, Jaesik Park. Qian- Yi Zhou, and Vladlen Koltun. Tanks and temples: benchmarking large-scale scene reconstruction. ACM Trans. Graph., 36(4), jul 2017.
[51] Zhewei Huang, Tianyuan Zhang, Wen Heng, Boxin Shi, and Shuchang Zhou. Real- saal-time intermediate flow estimation for video frame interpolation. In Shai Avidan, Gabriel Brostow, Moustapha Cissé, Giovanni Maria Farinella, and Tal Hassner, editors, Computer Vision - EECV 2022, pages 624-642, Cham, 2022. Springer Nature Switzerland.
[52] Daniel Gehrig, Mathias Gehrig, Javier Hidalgo-Carrió, and Davide Scaramuzza. Video to events: Recycling video datasets for event cameras. In IEEE Conf. Comput. Vis. Pattern Recog. (CVPR), June 2020.
[53] Zhou Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4):600-613 612, 2004.
[54] Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metrie. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018.
13

---

[55] Zichao Zhang and Davide Scaramuzza. A tutorial on quantitative trajectory evaluation for visual(-inertial) odometry. In 2018 IEEE/RSJ International Conference Intelligent Robots and Systems (IROS), pages 7244-7251, 2018.
[56] Jürgen Sturm, Nikolas Engelhard. Felix Endres, Wolfram Burgard, and Daniel Cremers. A benchmark for the evaluation rgb-d slam systems. 2012 JEEE/RSJ International Conference on Intelligent Robots and Systems, pages 573-580, 2012.
[57] Peng Wang, Yuan Liu, Zhaoxi Chen, Lingjie Liu, Ziwei Liu, Taku Komura, Christian Theobalt, and Wenping Wang. F2-nerf: Fast neural radiance field training with free camera trajectories. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4155041559, June 2023.
[58] Stepan Tulyakov, Danicl Gchrig, Stamatios Georgoulis, Julius Erbach, Mathias Gehrig, Yuanyou Li, and Davide Scaramuzza. Time lens: Event-based video frame interpolation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 16155-16164, June 2021.
[59] Lin Yen-Chen, Pete Florence, Jonathan T. Barron, Alberto Rodriguez, Phillip Isola, and Tsung- Yi Lin. inerf: Inverting neural radiance fields for pose estimation. In 2021 JEEE/RSJ International Conference on Intelligent Robots Systems (IROS), pages 1323-1330, 2021.
NeurIPS Paper Checklist
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction accurately reflect the main contributions. Please refer to the main body of the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long it is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
Justification: Please refer to Appendix A.9 for our discussions on the limita-
tions.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
14

---

The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't It acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proof's
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification: We provide the necessary formulas and related explanations in the main text. Please see refer to Method section about the blur model main paper.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: Justification: We identify our baseline method and explain the implementation
details where differ from the baseline method in Experiment Setup 5.3
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access to a hosted model (c.g., in the case
15

---

of a large language model), releasing of a model checkpoint, or other means that are
appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (c.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (c.g., to registered users), but it should be possible for other researchers have some path reproducing verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification: The datasets and baseline methods in this paper are publicly available. Our source code and the self-collect dataset will be openly accessible after the paper is accepted.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips... cc/
public//uuidee/Cdeattbbiiss Cerooooeesubmiiiooooooooooo ooolliicy))) for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/Ct////ooeeeeeeeeeeeeeeeeee ooo.liiiccoiiioooooooooooooo onPolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We have provided necessary implementation details in Section 5.3
Guidelines:
The answer NA means that the paper does not include experiments.
16

---

The experimental setting should be presented in the core of the paper to level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: We follow the experimental settings previous works to conduct experiments. Therefore, the comparison with previous works is fair.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (c.g. negative eтror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We have provided the information of our computer resources Section 5.2 and Section A.4.3
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https: http::::///ee..eeeeeeeeeeeeree http:///rrpp.ccppppc//ccccccoaaa
Answer: [Yes]
17

---

We ensure that our experiments comply with the NeurIPS Code of Ethics in all respects.
Justification:
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there is a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Please refer to our discussion on broader impacts in Section A.10
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks for misuse of released data models.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
18

---

12. Licenses for existing assets
Question:
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We have appropriately cited related work and abide by their licenses and terms. Please refer to Section|A.II
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g.. CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g.. website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of in the package should be provided. For popular datasets, paperswi wwwode....d..... com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided. this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: Justification: We detail the data acquisition in Appendix A.2
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
19

---

15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/teview based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
Declaration of LLM usage
Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: LLM is only used for writing.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should be described.
20

---

A Appendix
A.1 Discussion on Motion Blur
Image blur arises from brightness integration during exposure time and occasionally occurs in low- light conditions or high-speed motion scenarios. While our method does not explicitly target image blur, this does not diminish its core contributions. We can draw analogy to 2D vision tasks: image deblurring addresses blur-induced degradation, while video frame interpolation tackles discontinuity from excessive inter-frame motion. These represent distinct challenges in computer vision. Similarly, in 3D reconstruction, blur-related degradation and the challenges of sparse viewpoints with pose estimation constitute separate problem domains that often co-occur in high-speed scenarios but require distinct technical solutions. Our work specifically addresses the latter- cnteerfunddmentee issues that persist regardless blur conditions and represent critical bottlenecks in high-speed 3D reconstruction.
Previous event-based scene reconstruction methods have primarily focused blur reconstruc- tion while overlooking the challenges of sparse viewpoints and inaccurate pose estimation. Our approach targets this gap by leveraging a novel fusion strategy enhance traditional image- based 3DGS reconstruction, addressing fundamental limitations that affect reconstruction quality independent of blur artifacts.
For completeness, we conducted supplementary experiments to evaluate our method's performance under motion blur conditions. We extend our method to handle motion blur by incorporating the Event Double Integration (EDI) model [41], which reconstructs sharp intensity from event data. Specifically, we reformulate the intensity term 4,0 in Eq. (5) using the EDI formulation:
(2n + 1) B: Σx=-n
1.0 -i=-nexpp exp C. E.E Ei,3
(19)
where B; represents the blurred intensity, E.,, denotes the accumulated events, C is the contrast threshold, and n defines the temporal integration window of blur averaging.
performed multiple scenes from the Tanks and Tem- ples [50] with a frame rate of 2FPS. Motion-bluree frames were synthesized adopting the blur generation protocol from the GoPro-Blur dataset through gamma correction and multi-frame averag- ing operations. We average every 30 frames to simulate blurring. We selected EvDeblurNeRF [40), currently the best-performing open-source event-based deblur scene reconstruction method, comparison.
Experimental Setup: Experiments
Results: As demonstrated in Tab. 5 our method achieves comparable reconstruction quality to EvDeblurNeRF while simultaneously performing pose estimation. Visual comparisons in Fig, 7 show that our method renders sharper novel views under motion blur conditions. Note that SyDeblurNeRF requires pre-computed COLMAP poses. This limitation significantly restricts practical applicability in high-speed scenarios where accurate pose estimation is challenging. These results demonstrate that our method maintains robust performance under blur conditions while addressing the fundamental pose estimation and vier challenge.
Table 5: Comparison of different methods on pose estimation and rendering quality.
Methods
Pose Estimation
PSNRT
SSIM
RPEN
RPE,
EvDelurNeRR
CF-3DGS
21.17
0.69
0.105
0.897
EvDeblurNeRF
COLMAP
22.58
0.70
EDI + EF-3DGS (Ours)
EDI + EF-3DGS (Ours)
23.18
0.72
0.062
0.642
A.2 Dataset
For the synthetic dataset, following LocalRF g we choose nine static scenes from the Tanks and Temples dataset, which cover both indoor and outdoor environments. To construct a real-world dataset, utilized handheld DAVIS346 event camera to capture series of extended video sequences,
21

---

Blur Image
Events
COLMAP Ev Ev-DeblurNeRF
EDI EF-3DGS (Ours)
WEST
WEST TERN PAUIF IC
PARIFIC
PAUHE LC
Figure 7: Qualitative comparison motion deblur scene reconstruction. Our EDI EF-3DGS method produces sharper results comparable to EvDeblurNeRF
simulating free camera trajectories. Particularly for the scenes named building, deliberately introduced significant camera motion during acquisition to emulate realistic camera shake scenarios. Details about these sequences are illustrated in Table where Mean. rolation represents the mean relative rotation angle between two adjacent frames and Max. rotation denotes the maximum relative rotation angle between two frames in sequence. We select a 50-second segment from each sequence to highlight scenarios with free- fre--rajectory camera movements.
Table 6: Details of selected sequences Tanks and Temples [0 and RealEv-DAVIS.
Scenes
Seq. Length
Mean. rotation (deg)
Max. rotation (deg)
Auditorium
300
2.91
54.55
Tanks and Temples
Ballroom
300
5,73
179.81
Caterpillar
300
3.18
102.37
Church
300
2.44
37.22
Courtroom
300
8.40
177.96
M60
300
5.29
179.99
Museum
300
6.05
176.98
Panther
300
4.33
124.63
Train
300
4.80
108.18
RealEvDDAIS
building
500
5.65
67.65
hall
500
3,88
106.73
corner
500
3.04
96.11
outdoor
500
2.36
77.18
A.3 Overall Training Pipeline
We detail the comprehensive training pipeline Algorithm I Note that for notational simplicity, consolidate the subscripts (i.j) into single index k. Our approach incorporates the dynamic radiance field allocation strategy from LocalRF [8], which assigns new radiance field when the current pose exceeds a distance threshold of 1 from the existing field. However, in practical implementations, we encountered the OOt(out of memory) problem with this approach. To address this, we adopted modified strategy of allocating new 3DGS every 50 frames, based the sequence length. This adjustment ensures efficient memory utilization while maintaining the benefits of dynamic allocation. For pose optimization and scene reconstruction, adopt strategies from CF-3DGS.
22

---

And we incorporate event-driven brightness and motion coherence constraints, enabling robust reconstruction in challenging high-speed scenarios.
Algorithm 1 Overall Training Pipeline.
1: Input: event frames { k (k = =i. j).
2: Output: camera poses {Tk K 3DGS {Go.
3: Initialize current center frame of 3DGS,
4: while < do
Not finish the optimization yet [5]. initialize global 3DGS using the current center frame
5: Ge., + Initialize((i) > Following
I
6: for k in [t ml 50, K] do > Optimize the 3DGS and the poses of next 50 frames
7:
8:
# Pose Estimation Step
Giocal Initialize(Ig-))
Following
[ initialize local 3DGS using the previous
frame TK-1
10:
Th + OptiimzzeLeeeent Tk) estimate camera pose Ti by minimizing Levent at Tk
11:
end for
12:
13:
# Scene Reconstruction Step
{Go,} t OOpiimize(Ceeent T(,++50))
b Optimizing 3DGS Gg, by minimizing Cevent at
Tl.1+50j
15:
SHs of (Ga., + Optimize(fccttor T(.++50)
& Fixed-GS stage: optimizing SHs of 3DGS
Go., by minimizing Ceotor RGB frames
16: 1 k
> Slide the current center frame to the next center frame
end while
Table Ablation study results of parameter r CMax framework
NVS
Pose
PSNR f
SSIM
LPIPS
RPE,
RPE, +
ATE +
22.94
0.70
0.39
0.0591
0.6829
0,4851
3
23.96
0.72
0.36
0.0487
0.6259
0.3753
5
23.49
0.72
0.37
0.0524
0.6431
0.4338
A.4
Additional Experiments
A.4.1 Detailed Experiment Results RealEy-DAVIS
Due space constraints in the main text, we provide detailed table of method's performance on the RealEy-DAVIS dataset in the supplementary materials, as shown in Tab. and Tab.
Table 8: Quantitative Evaluations on RealEy-DAVIS. We select the top-performing methods from the previous evaluation. Each baseline method trained with its public code under the original settings and evaluated with the same evaluation protocol. The best results are highlighted in bold.
Viluab
Pow
22.11
031
11.3
ECH4
0.3
E42
2442
1.3
12.0
655
8..0
$4.00
K50
1551
0.38
Guach
Per Eics
Ext
11.5
64Mh
6.4
641
179
611
680
1400
8.11
0.001
11:34
A.4.2 Evaluating the Influence of parameter in LEGM
We investigate the impact of the parameter in the Contrast Maximization (CMax) framework, which determines the number of previous event frames warped to the current sampled timestamp. Table presents the results of this ablation study. The optimal value for is found to be 3, yiclding the best
23

---

Table 9: Pose accuracy RealEv-DAVIS. The unit of RPE, is in degrees, ATE is in the ground truth scale and RPE, is scaled by 100. Those methods which require precomputed poses are excluded.
2301
23835
3.000 9440 61100
11777
8.90
M8P0 8504
K599
3.796
0309
performance across all metrics for both novel view synthesis and pose estimation. When 1, the performance degrades significantly, likely due to insufficient temporal information for accurate motion estimation. Increasing r to 5 leads to a performance decline, though less severe than = 1. This degradation r can be attributed to the violation of the local linear motion assumption, which is fundamental to the CMax framework.
A.4.3 Computational Efficiency and Impact of Subinterval Number N
To investigate the impact of the number of subintervals N on our method, we conduct ablation studies on the Tanks and Temples dataset under the FPS setting. The results are shown in Table TO and Fig. 8 We test the speed on RTX2080ti. Note that in the t-PSNR figure in Fig 8 circular nodes represent pose-free methods, while diamond-shaped nodes indicate methods that require precomputed poses. The results in Table I0 reveal that increasing N improves PSNR, indicating that finer temporal resolution enhances reconstruction quality. However, this improvement comes at the cost of an extended training time, increasing from 1.3 hours at N=2 to 7 hours N=6. Importantly, method maintains real-time rendering performance (30+ FPS), matching CF-3DGS as both utilize the same efficient 3DGS rendering framework. This gives over slower NeRF-based methods. Even at N=2, our approach outperforms baselines in both PSNR and rendering speed. Event-3DGS achieves a good balance of efficiency and performance, but it requires precomputed poses. The setting of N=3 offers a good balance, achieving higher PSNR than CF-3DGS with comparable training time. However, it's important note that the optimal N may vary for different frame rates and motion characteristics.
approach significant advantage
Table 10: Effect of the number of event subintervals N.
PSNRT
FPS
tl
= 2
23,56
30+
1.3h
15
N 3
24.43
30+
3h
Trannin
N 6
24.81
30+
7h
Lnckis
CE-EDOS
LocalRF
21.20
<1
8h
D.Delarrhed
CF-3DGS
22.08
30+
3h
Figure 8: Comparison of training time vs PSNR for various methods
PSNR
ATE
0.8
PSNR
ATE
0.8
24
0.6
PSNR
-33
0.4
0.4
0.4
22
22
0.2
0.0
0.1
Amm
Ages
Figure 9: Study of the differ- ent evalution metrics with re- spect to Xem
Figure 10: Study of the dif- ferent evalution metrics with respect Agrad
Figure 11: Study of the dif- ferent evalution metrics with respect to XPBA
24

---

Trunding View
Dept
Beckes
wife Lucee
w/ 4a
Sep WY Fendos
Figure 14: Qualitative Results of the effective- ness of LPBA-
Figure 12: Qualitative Results of the effectiveness of LEGM-
Figure 13: Qualitative Results of the effectiveness LLEGM-
A.4.4 Analysis of Loss Coefficients
Different components of our method play crucial roles in overall performance. To comprehensively understand these effects and determine optimal weight settings, we conducted a detailed analysis of the loss weights: Aem. Agrad and APBA-
Contrast Maximization Coefficient Xem As shown in Fig. 9 the contrast maximization coefficient Acm significantly affects both NVS quality and pose estimation. We observe that a value of 0. achieves the best overall performance. Lower values (0.01) lead to decreased performance, likely due to insufficient utilization of motion information from events. Higher values (0.5) also result in performance degradation, possibly due to over-reliance on the contrast maximization term at the expense of other constraints. The optimal Acm suggests that while event data is crucial, it should not dominate the reconstruction process. This balance allows our method to leverage the high temporal resolution of events without sacrificing the global supervision provided by frame data.
Gradient-based Loss Coefficient Agrrad As shown in Fig. 10 the gradient-based loss coefficient X Agrad shows optimal performance 0.2. Lower values (0.1) slightly decrease performance, while higher values (0.3, 0.5) lead to more significant drops in both NVS quality and pose accuracy. This is likely because LEGM and 3DGS primarily focus on rendering absolute brightness, and excessive gradient constraints may interfere with this process. Therefore, moderate Agrod value is crucial to balance gradient information with the primary rendering objectives.
Photometric Bundle Adjustment Coefficient λpbA As illustrated in Figure I1 our method demon- strates remarkable stability across λPBA values ranging from 0.1 to 0.5. However, while CPRA has minimal effect on PSNR and ATE metries, its absence leads to noticeable color distortion artifacts in visual results. This underscores the importance of λрвА in maintaining visual fidelity. Conversely, setting APBA to 1.0 results in significant performance degradation, indicating that overemphasis on cross-view consistency can be counterproductive.
A.5 Visualization of Ablation Studies
Due to space constraints, provide comprehensive visualizations of our ablation studies in this section. Our visualizations demonstrate that while CF-3DGS can produce high-quality renderings at training views, it struggles with adjacent novel views. The rendered depth maps expose considerable inaccuracies in scene geometry reconstruction. With LEGM, as shown in Fig. 12 we observe marked improvements it effectively harnesses continuous brightness change data captured by event cameras, compensating for inter-frame information loss. For LLEGM. as shown in Fig. [3], our visualizations show that without this component, the generated Image of Warped Events (IWE) exhibits blurring effects, indicating inaccurate pose estimation. This is particularly evident in self-similar regions with repetitive textures, which pose additional challenges. LLEGM optimizes IWE sharpness, effectively improving both rendering quality and pose estimation. Regarding LPBA and Fixed-GS. as shown in Fig. [4] our visualizations reveal that without these components, reconstructed scene loses almost all color information. This occurs because event- -uidee optimization, with substantially more event frames than image frames, dominates scene reconstruction and suppresses color information. CPBA alone, which reprojects neighboring video frames onto event frames, only partially addresses this imbalance. The Fixed-GS strategy separates color and structure optimization, and when combined with PBA, they complement each other to effectively resolve color distortion issues.
25

---

A.6 Additional Challenging Scenario Experiments
To further demonstrate event camera advantages, we constructed FASTER scenario by selecting every 10th frame from the SLOW dataset and directly applied COLMAP pose estimation on these challenging sequences. We added COLMAP+3DGS as comparison baseline. Notably, in the corner and outdoor scenes, several frames exhibit registration failures where the estimated poses greatly deviate from the overall video trajectory, indicating COLMAP's limitations in challenging real-world scenarios.
Table 11: Rendering quality comparison across different speed scenarios on RealEv-DAVIS dataset.
SLOW
FAST
FASTER
Methods
PSNR
SSIM
PSNR
SSIM
PSNR1
SSIM
CF-3DGS
22.68
0.629
17.59
0.520
15.61
0.504
COLMAP+3DGS
23.27
0.633
19.79
0.548
17.25
0.521
EF-3DGS (Ours)
23.65
0.647
21.12
0.562
20.18
0.532
As shown in Table ID the results demonstrate that: (1) COLMAP+3DGS rendering quality degrades significantly as motion speed increase confirming pose estimation inaccuracies in challenging con- ditions. (2) Compared to CF-3DGS, our method maintains substantially better rendering quality in high-speed scenarios, highlighting our method robustness and demonstrating the potential of event cameras in high-speed scenarios.
A.7 Necessity of Two-Stage Optimization
To validate the necessity of our two-stage Fixed-GS training strategy, we explore an alternative approach: incorporating a reference view rendering loss Get at each training iteration to address the color distortion challenge without requiring separate optimization stage.
Given that events have much higher temporal resolution than RGB frames, the sparse color constraints from RGB images are largely overwhelmed by the abundant grayscale constraints from event data, ultimately resulting in color distortion. By incorporating fres, the backpropagated gradients each iteration would be augmented with color information from the reference view, potentially alleviating the color distortion issue. We conducted additional experiments comparing this approach with our two-stage strategy. As shown in Table 12 the results show that incorporating Leef achieves comparable rendering quality. Although we cannot include visualizations here, our qualitative results confirm that the color distortion issue is effectively alleviated with this approach. However, this comes at significant computational cost, increasing training time (3.8h vs 2.5h) due to additional reference view rendering at each iteration. Considering the modest performance difference relative to the substantial training overhead, we believe the two-stage optimization strategy offers a more efficient and practical solution.
Table 12: Comparison of different color correction strategies.
Method Description
PSNR
SSIM
LPIPSJ.
Training Time..
Baseline
23.09
0.70
0.38
2.7h
Levent + Lcotor
24.11
0.73
0.36
3.8h
Levent two-stage (Fixed-GS) strategy
23.96
0.72
0.36
2.5h
A.8
Additional Visualization
We present additional qualitative results on both Tanks and Temples and RealEy-DAVIS.
A.9
Limitations
Our method combines event streams with conventional video frames, which require the input data to be time-ordered. So it can not handle the unordered image input. And, our method relies on several
26

---

key parameters (e.g,, Aem. Agrad and XP that may require manual tuning for optimal performance across different speed scenarios. This dependency on scene- seen--pecific parameter settings could limit the method's adaptability to diverse environments. Future work should explore self self-adaptive parameter adjustment strategies to enhance the method's versatility and ease of use across various reconstruction tasks.
A.10
Broader Impacts
To the best of our knowledge, the proposed method will have significant negative social impact. The proposed scene reconstruction method can be used to reconstruct and render some wild scenes. Users can use the video shot by their mobile phones input to obtain an explicit 3D asset represented by a 3D Gaussian. This 3D asset can be used for subsequent editing, development, secondary creation for entertainment.
A.11 Data Availability
The datasets that support the findings of this study are available in the following reposito- ries: Tanks and Temples [50] at lnttps://ww. tanksandtemples, org/ under CC BY 4.0 li- cense. The code of choosen baseline. Nope-NeRF [00] is available at https://githhu: com/ CCVCL//BDD-Gausiiaan under Apache-2.0 license, LocalRF is available https:/ localrf. 1o/ under Apache-2.0 license, Event-3DGS
44 is available at https: /
Bgtaabb.cco//aappppparreee-----S under Apache-2.0 license, EvDeblurNeRF [40] is available
under Apache-2.0 license, ENeRF [36] is avail- under MIT license, CF-3DGS is available at http:://ggthbb..ccm///aabbb////s under Apache-2.0 license,
able at halha/pps//iihhu.ccc/////e//e/ee
27

---

CF-3DGS
Event-3DGS
EvCF-3DGS
EF-3DGS(OUrs)
Ground-frumt
Figure 15: Qualitative comparison for novel view synthesis on Tanks and Temples Our approach produces more realistic rendering results with fine-grained details. Better viewed when zoomed in.
28

---

CF-3DGS
Event-3DGS
EvCF-3DGS
EF-3DGS(OUrs)
Ground-truth
Figure 16:
Qualitative comparison for novel view synthesis on RealEv-DAVIS. Our approach produces more realistic rendering results with fine- iinengrieedd details. Better viewed when zoomed in.
29

---

LocaRF COLMAP(OT)
CF 3DGS COLMAPIOT)
ExCF 3DGS OOLM
OuTs
1.0
COLMAPIGT)
0.8
Audiorium
0.6
0.4
&
-.2
0.0
Frama ID
Frame ID
Frame ID
Frame ID
LocaRF COLMAPICT
3DGS COLMAPIOT)
EvCF 3DGS COLMAPIGTT
Ours
COLMAPYGT)
Belfcom
&
Frame
Frame
Frame
Frame ID
LocaRF COLMAPICT)
CF 3DGS COLMAPYOT)
ExCF 3DGS COLMAP(CTT
COLMAPICT)
Outs
1.0
0.8
0.6
0.44
&
-02
-.0
Frame ID
Frame ID
Frame ID
Frame ID
Loca COLMAP(CT)
3D-3 COLMAPYGTI
EvCF 3DGS COLMAPIOTT
COLMAPYCT)
Caterpttor
&
Frame
Frame
Frame ID
LocaRF COLMAPICT)
CDLMAPYGT)
CF 3DG3
EvCF 3DGS COL MAP(OT)
Ours
COLMAP(CT)
1.0
0.8
Courtroom
0.8
0.4g 2
0.2
-.0
Frame ID
Frame ID
Frame ID
Frame ID
Figure 17: Pose Estimation Comparison on Tanks and Temples. We visualise the trajectory (3D plot) and relative rotation errors RPE, (bottom colour bar) of each method. We clip and normalize the RPE, by quarter of the max RPE, across all results of each scene.
30

---

COLMPPIGT)
(OLMPP/GT)
COLMPPIST)
COLMAPIGT)
0.6
M60
0.4
2
0.2
0.0
Frame
Frame
Frame
Frame ID
LocaRF COLMAPIGT
GF 30335
ExGF 3D0GS COLMAPIGIT
Ours
1.0
COLMAPYOT
COLMAPIGT)
0.8
Moseum
0.6
0.4
2
Frama ID
Frame ID
Frame ID
Frame ID
LF COLMAP(OT)
COLMAPYOT)
3F-306S
3DGS COLMAP(OT)
COLMAPIOT)
Ours
1.0
0.8
0.8
Panther
0.4g
0.2
0.0
Frame
Frame
Frame ID
Frame ID
Loca/RF
COLMAPIGT)
CF-30GB
COLMAP/GT)
EVCF COLMAR(GIT
Ours
COLMAPIGT)
1.0
0.8
0.4
Frame
Frame
Frame ID
Frame ID
Figure 18: Pose Estimation Comparison on Tanks and Temples. We visualise the trajectory (3D plot) and relative rotation errors RPE,- (bottom colour bar) of each method. We clip and normalize the RPE, by a quarter of the max RPE, across all results of each scene.
31

---

COLMAPIGT)
COLMPPIGT
COLMMP(STT
COLMAPIGT)
0.4
22
0.2
0.0
Frame
Frame
Frame ID
Frame ID
LocaFFF
GF GF-3095
EXGF JOOS
Ours
1.0
COLMAPIOT)
COLMAP(GIT
COLMAPIGT)
0.8
0.6
0.4
2
Frame
Frame
Frame ID
Frame ID
COLMAPIOT)
COLMAPYOT)
COLMAPIOT
COLMAPIOT)
1.0
0.4
0.2
0.0
Frame
Frame
Frame ID
Frame ID
Loca/RF
CF-3DG8
EVCF-3DG8
Olrs COLMAPIGT)
COLMAP(GIT
0.4w
Frame
Frame
Frame ID
Frame ID
Figure 19: Pose Estimation Comparison on Tanks and Temples and RealEv-DAVIS, We visualise the trajectory (3D plot) and relative rotation errors RPE, (bottom colour bar) of each method. We clip and normalize the RPE, by a quarter of the max RPE, across all results of each scene.
32


Reviews:
[{"review_id": "w82XFmvQ37", "reviewer": "Reviewer_cYjR", "summary": "This paper proposed EF-3DGS. It is a 3DGS model based on 1. high-frequency event camera stream 2. RGB video frames and 3. no trajectories aka camera poses. The overall problem setup is compelling since event camera captures a continuous stream that encapsulate fine-grained motion and RGB video frames contains rich visual details for constraining the overall structure. Such a setup without any presumption on trajectory or poses is worth studying. This paper integrates the principle of EGM, CMax and LEGM into this setup. Further, the proposed PBA refines the continuous trajectory supervised by sparse video frames. The overall pipeline is split into two stage where the first stage optimize the overall geometry and monochrome color and second stage refines the color with RGB video stream. Results show effectiveness on both reconstruction quality and pose/trajectory estimation.", "questions": "Such setup is quite commonly observed on robotics and autonomous vehicles. I would like to know you thoughts on how far away from EF-3DGS to real-time, online processing, since inherently your work does not need any trajectory. Is it straight forward or still unanswered questions?\n\nminor suggestions:\n\n- place figure 3 and figure 6 in a better position, somewhere closer to it is first mentioned.\n- some basic introduction around EGM, CMax and LEGM might go to the preliminary sections. Otherwise it is confusing which part in the methodology is the contribution of this work.\n- It can also be interesting to visualize the optimization process, that as loss being optimized, the warped image gets sharper and trajectory is optimized simultaneously.", "limitations": "yes", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength\n\n- The  proposed pipeline which integrates different modalities, 3DGS, event generation model and bundle adjustment, is overall sound and reasonable.\n- Sufficient experiments that both presents results quantitatively and qualitatively, with visualizations and numerical comparisons. Results show good performance on both reconstruction and trajectory recovery.\n\nweakness\n\n- Figure 3 might be too small can hardly tell much from the comparison (but the one in the appendix is ok so not a big issue). The location of figure 3 need adjust as well.\n- With the loss mentioned in eq.18, both rgb and grayscale loss are involved. It is unclear how they are optimized together.\n\nMethodology part need some reorganization:\n\n- on page 5 where IPWE is introduced, it is illustrated in Figure 6 but Figure 6 is on page 9.\n- in 4.4, the argmin equation is minimizing $\\mathcal{L}_{event}$ but this loss is not introduced until 4.5.", "was_revised": true, "final_justification": "I remain the opinion that this paper is a qualified contribution to NeurIPS and scores are remained as well.", "rebuttal": "We thank the reviewer for the constructive feedback and for recognizing our compelling problem setup, sound pipeline, and sufficient experimental validation. We respond to each concern in detail. Please feel free to use the discussion period if you have any additional questions.\n\n> ### **W1 & W3 & W4 & Minor S1: Paper format and reorganization issues.**\n\n**Answer:** We apologize for the confusion caused by the figure size, figure placements and ordering issues in our methodology section. We will address these issues in the revision by adjusting the figure size, moving figures closer to where they are mentioned, and ensuring all concepts are defined before they're used. Thanks for bearing with these organization issues.\n\n> ### **W2: Confusion about loss optimization in Eq. 18.**\n\n**Answer:** We clarify that RGB and grayscale losses can be optimized simultaneously. For grayscale losses ($L_{EGM}$ + $L_{LEGM}$), we convert 3DGS RGB renderings to grayscale using $Y = 0.299 \\cdot R + 0.587 \\cdot G + 0.114 \\cdot B$, then compute the losses. The RGB loss ($L_{PBA}$) is computed directly on the original RGB renderings. For conciseness in our mathematical formulation, we did not explicitly indicate the color space for each loss computation. We will clarify this in the revised version.\n\n> ### **Q1: Potential of EF-3DGS for real-time and online applications.**\n\n**Answer:** \n\n​\tThanks for your interest in the real-time feasibility of our approach, which is indeed crucial for robotics and autonomous driving applications. We respond from two aspects: 1) Current performance analysis and 2) future directions.\n\n​\t**1) Current performance analysis**: We have provided a comprehensive computational analysis in Appendix A.7. For rendering speed, our method achieves over 30 FPS, approaching real-time performance. For training time, our approach achieves significantly better reconstruction quality than previous pose-free methods (CF-3DGS, LocalRF) without substantially increasing computational overhead. However, we acknowledge that all current pose-free methods, including ours, require per-scene optimization, making real-time reconstruction still challenging. \n\n​\t**2) Future directions**: Recent work VGGT[1] demonstrates promising directions by utilizing large Transformers to regress reconstructed point clouds and poses within seconds from input images. We believe combining the VGGT paradigm with EF-3DGS presents tremendous opportunities, though several challenges remain: 1) How to extend point cloud reconstruction to 3DGS reconstruction. 2) How to enhance model performance in high-speed (sparse view) scenarios. 3) How to train a unified model receiving both events and images as inputs, even if they have significant data modality differences.\n\n​\tWe are excited about these directions and believe they represent the next frontier for real-time and pose-free scene reconstruction. As you noted, these remain challenging questions. However, we view them as promising avenues for future research.\n\n> ### **Minor S2: Methodology organization of EGM, CMax, and LEGM.**\n\n**Answer:** Thank you for your suggestion. We agree that a better organization would significantly improve clarity and better distinguish our contributions. We will reorganize the methodology section for better clarity. We plan to move EGM part to the preliminary section as it describes the fundamental working principle of event cameras. CMax and LEGM parts will remain in the methodology section as they represent the core components of our approach: extracting motion information from event streams for pose optimization, and providing gradient-domain constraints through warped events for well-constrained scene reconstruction.\n\n> ### **Minor S3: Visualization of the optimization process.**\n\n**Answer:** Thanks for your suggestion. We have actually provided trajectory estimation visualization in our supplementary materials (videos), which shows the pose estimation process as video frames and event streams are progressively input. However, due to current rebuttal format constraints, we plan to include the specific visualization of warped events during optimization in the future revised version.\n\n\n\n[1] Wang, Jianyuan, et al. \"Vggt: Visual geometry grounded transformer.\" *Proceedings of the Computer Vision and Pattern Recognition Conference*. 2025."}, {"review_id": "SDARVsTXCS", "reviewer": "Reviewer_hsxP", "summary": "This paper introduces the event cameras to aid scene construction from a casually captured video for the first time, and propose Event-Aided Free-Trajectory 3DGS, called EF-3DGS, which seamlessly integrates the advantages of event cameras into 3DGS through three key components", "questions": "Please see the weakness.", "limitations": "See the weakness.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths\n\nClarity and Visualizations: The paper is exceptionally well-written, and the figures are clear and easy to understand.\n\nRich Content: The overall content presented in the paper is comprehensive and substantial.\n\nWeaknesses: \n\n1. Necessity of Two-Stage Optimization: The paper proposes a two-stage framework: first optimizing the shape of Gaussian spheres for texture features, and then optimizing colors. While this idea is novel, its necessity is debatable. Specifically, in Section 4.1 (EGM optimization), could the optimization directly use the given reference image plus event data? For example, by calculating the difference between the 3DGS-rendered image and the reference image, and then using this difference with event data as a loss function. Combined with the authors' introduced optical flow warping, could this directly address the color issue? I would be very interested in seeing an ablation study on this specific aspect.\n\n2. \"Pose-Free\" Claim Discrepancy: The paper's title claims a \"pose-free\" approach, yet the methodology section lacks any explicit description or discussion of how this pose-free aspect is achieved. Is this work directly built upon an existing pose-free 3DGS method, or is there a novel contribution to pose estimation that is not clearly articulated?\n\n3. Justification for Pose-Free Approach: The necessity of a pose-free method needs stronger emphasis. A detailed discussion on how inaccuracies in pose estimation in real-world scenarios significantly impact other methods would strengthen the paper's claims and highlight the importance of their \"pose-free\" contribution.", "was_revised": true, "final_justification": "Authors have solved my problem and i raise my score to accept, but my main concern is that this two-stage optimization strategy is not elegant enough and i believe there will be potential solutions.", "rebuttal": "We thank the reviewer for the constructive feedback. We appreciate the recognition of our paper's writing and comprehensive content. We respond to each concern in detail below, and welcome any additional questions during the discussion period.\n\n> ### **W1: Necessity of Two-Stage Optimization**\n\n**Answer:** We acknowledge that our method description may have caused some confusion, and we would like to address this concern from two perspectives: 1) clarification of our current approach and 2) exploration of the alternative solution suggested by the reviewer.\n\n**1) Clarification of EGM Mathematical Equivalence**: The reviewer's proposed solution is actually mathematically equivalent to our EGM formulation. The suggested loss function can be expressed as:\n$$\\min~||\\frac{\\hat{I}\\_t}{I\\_{i,0}} - \\exp{(\\sum\\_{i=1}^{n} E\\_{i,n}\\cdot C)}||,$$\n\nwhere $\\frac{\\hat{I}\\_t}{I\\_{i,0}}$ represents the difference between rendered and reference images, and $\\exp{(\\sum\\_{i=1}^{n} E\\_{i,n}\\cdot C)}$ corresponds to the brightness changes recorded by event data. By multiplying both sides by $I\\_{i,0}$, the loss function becomes:\n$$||\\hat{I}\\_t - I\\_{i,0}\\cdot\\exp{(\\sum\\_{i=1}^{n} E\\_{i,n}\\cdot C)}||$$\nAccording to our Eq. 5, $I\\_t = I\\_{i,j} = I\\_{i,0}\\cdot\\exp{(\\sum\\_{i=1}^{n} E\\_{i,n}\\cdot C)}$, which transforms the objective to \n$$||\\hat{I}\\_t - I\\_t||=L\\_1(\\hat{I}\\_t, I\\_t).$$\nThis demonstrates the mathematical equivalence between the reviewer's suggestion and our EGM formulation.\n$$\\min||\\frac{\\hat{I}\\_t}{I\\_{i,0}} - \\exp{(\\sum\\_{i=1}^{n} E\\_{i,n}\\cdot C)}||\\Leftrightarrow\\min||\\hat{I}\\_t - I\\_{i,0}\\cdot\\exp{(\\sum\\_{i=1}^{n} E\\_{i,n}\\cdot C)}|| \\Leftrightarrow \\min||\\hat{I}\\_t - I\\_t||.$$\n\nSince event data lacks color information, the loss computation must be performed in grayscale space, resulting in gradients that also lack color information. Therefore, the reviewer's suggested formulation cannot address the color distortion problem.\n\n**2) Exploration of Alternative Approach**: Although the reviewer's proposed approach is mathematically equivalent to our EGM, inspired by the reviewer's constructive feedback, we investigated whether adding a reference view rendering loss could address this challenge:\n$$L\\_{color} = (1 - \\lambda)L\\_1(\\hat{I}\\_{i,0}, I\\_{i,0}) + \\lambda L\\_{D-SSIM}(\\hat{I}\\_{i,0}, I\\_{i,0}).$$\nGiven that events have much higher temporal resolution than RGB frames, the sparse color constraints from RGB images are largely overwhelmed by the abundant grayscale constraints from event data, ultimately resulting in color distortion. By incorporating $L\\_{color}$, the backpropagated gradients at each iteration would be augmented with color information from the reference view, potentially alleviating the color distortion issue. We conducted additional experiments comparing this approach with our two-stage strategy:\n| Method Description | PSNR↑ | SSIM↑ | LPIPS↓ | Training Time↓\n|------------|-----|-----|-----|------|\n| $L\\_{event}$   | 23.09| 0.70| 0.38|2.7h|\n| $L\\_{event} + L\\_{color}$   | **24.11** | **0.73** | **0.36** |3.8h|\n| $L\\_{event}$ + two stage(Fixed GS) strategy   | 23.96| 0.72| **0.36**|**2.5h**|\n\nThe results show that incorporating $L\\_{color}$​ achieves comparable rendering quality.\nAlthough we cannot include visualizations in this rebuttal, our qualitative results confirm that the color distortion issue is effectively alleviated with this approach.\nHowever, this comes at a significant computational cost, increasing training time (3.8h vs 2.5h) due to additional reference view rendering at each iteration. Considering the modest performance difference relative to the substantial training overhead, we believe the two-stage optimization strategy offers a more efficient and practical solution.\n\n**Summary:** We appreciate the reviewer's insights. We proved the mathematical equivalence between the reviewer's suggestion and our EGM formulation. Inspired by the reviewer's feedback, we explored a viable alternative approach. However, our experiments demonstrate that our two-stage strategy is efficient and effective.\n\n> ### **W2: \"Pose-Free\" Claim Discrepancy**\n\n**Answer:** We thank the reviewer for this important clarification request. We sincerely apologize for not clearly explaining our pose-free implementation in the methodology section, as we focused primarily on introducing the event camera integration while overlooking the description of our overall pose-free pipeline. We address this concern in two parts:\n\n**1) How Pose-Free is Achieved**: Our pose-free approach is implemented through a progressive joint optimization framework detailed in Section 4.5 \"Overall Training Pipeline\" and Algorithm 1 (Appendix A.3). Specifically, we build upon the optimization scheme of CF-3DGS. As shown in Algorithm 1 in the appendix, we progressively optimize camera poses for each timestamp following the temporal order of video frames using our proposed event-driven loss $L_{event}$. During this process, we alternate between optimizing camera poses and reconstructing the 3DGS. Our two-stage Fixed-GS training strategy (Section 4.4) is integrated within the scene reconstruction step to mitigate color distortion issues. To handle long video sequences and OOM(out of memory) problem, we adopt LocalRF's local radiance field allocation strategy, allocating a new 3DGS every 50 frames.\n\n**2) Novel Contribution Clarification**: While we build upon CF-3DGS's optimization framework, our core contribution is not proposing an entirely new joint pose-scene optimization method. Instead, our novelty lies in effectively leveraging event camera properties to address two fundamental challenges that existing pose-free methods face in high-speed scenarios: (1) sparse scene reconstruction due to insufficient frame observations, and (2) inaccurate pose estimation caused by large inter-frame displacements. Our event-driven optimization provides dense supervision through EGM and extracts motion information via LEGM. The experiments across different frame rates (6FPS to 1FPS) validate that our performance advantage becomes greater as camera speed increases. Additionally, to address the color distortion issue introduced by event integration, we propose a simple but effective Fixed-GS two-stage optimization strategy that separates structure and color optimization.\n\nWe will pay more attention to the method description, ensuring that the pose-free implementation is well explained and its significance is highlighted.\n\n\n> ### **W3: Justification for Pose-Free Approach**\n\n**Answer:** We thank the reviewer for this valuable feedback. Pose-free methods are indeed crucial because traditional 3DGS methods rely on COLMAP for pose estimation, which frequently fails in challenging scenarios such as low-texture regions and high-speed motion. Nope-NeRF has demonstrated this issue through examples of COLMAP failure cases [1], validating the necessity of pose-free approaches. In fact, pose-free NeRF and 3DGS represent a very active research direction, with continuous developments, including BARF, Nope-NeRF, LocalRF, and CF-3DGS.\n\nWe apologize for not sufficiently emphasizing the critical issue of pose estimation degradation in high-speed scenarios. Our experiments demonstrate that existing pose-free methods suffer from severe pose estimation errors when camera motion becomes rapid. As shown in Table 2, on Tanks & Temples, as frame rate decreases from 6FPS to 1FPS, our method maintains significantly better pose accuracy (40% lower ATE) compared to CF-3DGS. Similarly, Table 3 shows superior robustness in FAST scenarios on RealEv-DAVIS. Our supplementary video provides visual evidence. The video clearly shows that CF-3DGS produces severe rendering artifacts (scene discontinuities) in high-speed scenarios due to inaccurate pose estimation, while our event-aided approach maintains stable pose estimation and high-quality rendering. Furthermore, we conducted supplementary experiments to validate the pose estimation inaccuracy issues of COLMAP and other pose-free methods in high-speed scenarios. Please refer to the \"**Additional Challenging Scenario Experiments**\" section in our response to **Quality 2** of **reviewer UuWp's** reviews.\n\nWe will revise the paper and add a detailed discussion to better highlight this critical contribution and its significance in the context of existing pose-free methods.\n\n[1] Bian, Wenjing, et al. \"Nope-nerf: Optimising neural radiance field with no pose prior.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023."}, {"review_id": "pUQBZK1ot1", "reviewer": "Reviewer_UuWp", "summary": "This paper proposes a novel method for 3D Gaussian Splatting of static scenes using images and event streams as input, while simultaneously optimizing the camera trajectory. The input images are temporally sparse, whereas the events provide dense temporal information. The proposed method consists of two stages. In the first stage, the parameters of the 3D Gaussians and the camera trajectory are jointly optimized using a loss function derived from the Event Generation Model (EGM), Contrast Maximization (CMax), and Photometric Bundle Adjustment (PBA). Since events do not contain information about absolute brightness values, accurate color reconstruction is not feasible in this stage. Therefore, in the second stage, only the color of the 3D Gaussians are refined using the RGB images.\n\nIn the experiments, the authors used the publicly available Tanks and Temples dataset, as well as the RealEv-DAVIS dataset constructed by the authors, to quantitatively and qualitatively evaluate the effectiveness of the proposed method. Additionally, the authors assessed the robustness of the proposed method to noise in the initial camera poses and demonstrated that it maintains strong performance under high noise levels compared to existing approaches.", "questions": "1. Contrast Maximization optimization is generally computationally expensive. In this paper, instead of computing the Image of Warped Events (IWE) for each individual event, the authors propose computing the Image of Piece-wise Warped Events (IPWE) by warping events that occur within a predefined time interval collectively. How much does this modification reduce the overall optimization time?\n2. In Section 4.4, the authors state, “The ratio between the first and second stages is empirically set to 4:1.” What does this ratio refers to? Does it indicate the ratio of the number of iterations used in the optimization for each stage?\n3. What is Ev-Baseline in Table 8?\n4. How is the initial camera poses given?", "limitations": "yes.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Quality**\n1. Questions remain regarding the experimental assumptions. The authors simulate large inter-frame displacements by reducing the number of frames. However, high-speed camera motions would typically result in motion blur in the captured images. The experiments in the paper assume that sharp, blur-free images are provided as input, which may not accurately reflect real-world conditions.\n2. In the RealEv-DAVIS dataset, COLMAP is used to obtain ground-truth camera poses, which implies that accurate camera pose can be estimated from images. To more clearly demonstrate the advantages of event cameras, it may be beneficial to conduct additional experiments where pose estimation from images is challenging or unreliable.\n3. Although the experiments in this paper assume that the threshold $C$ at which events are triggered is known, this assumption may not hold in general.\n\n**Clarity**\n1. The process of the proposed method is very clearly explained in the paper and is easily understood.\n2. The authors present extensive quantitative and qualitative experimental results, which effectively demonstrate the efficacy of the proposed method and aid the reader's understanding.\n\n**Significance**\n1. This paper serves as an example demonstrating the advantages of event cameras over frame-based cameras in 3D Gaussian Splatting. However, as previously noted, questions remain regarding the experimental setup, and the full potential of event cameras has not been comprehensively evaluated.\n\n**Originality**\n1. The authors leverage the unique characteristics of events by integrating frame and image information through the EGM, and by utilizing motion information encoded in the events for optimization via Contrast Maximization.", "was_revised": true, "final_justification": "The authors provided sufficient experimental results in their rebuttal, which led me to revise my previous rating.\nI believe that if the results of the experiments conducted by the authors in their rebuttal are incorporated into the final version of the paper, it will be worthy of acceptance by NeurIPS.", "rebuttal": "We thank the reviewer for the constructive feedback. We respond to each concern in detail. Please feel free to use the discussion period if you have any additional questions.\n\n> ### **Quality 1: Questions Regarding the Experiment Assumptions.**\n\n**Answer:** We sincerely appreciate the reviewer's insightful observation regarding motion blur. We respond to this concern from two perspectives:\n\n**1) Discussion on Motion Blur**: We acknowledge that motion blur and large pixel displacements may simultaneously occur in high-speed scenarios. To address the reviewer's concern about motion blur in our experimental setup, we would like to clarify our experimental setup and novel contribution through an established paradigm in computer vision.\n\nIn low-level vision, image deblurring and video frame interpolation represent fundamentally different challenges: image deblurring addresses degradation **within individual frames** during exposure time, while video frame interpolation tackles discontinuity **between different frames** due to large pixel displacements. Notably, video frame interpolation methods typically do not consider motion blur in their evaluation.\n\nDrawing from this paradigm, we can view 3D reconstruction similarly: our work specifically targets sparse viewpoint and pose estimation challenges arising from large pixel displacements between frames, while motion blur represents intra-frame degradation—a separate problem domain. Previous event-based 3D methods have focused on **intra-frame** blur mitigation assuming accurate poses. In contrast, we address fundamental **inter-frame** challenges, specifically joint pose-scene optimization and sparse view reconstruction, which are fundamental robustness challenges in 3DGS reconstruction.\n\n\n**2) Additional Experimental Validation**: For completeness, we conducted additional experiments to evaluate our method's performance under motion blur conditions. We extend our method using the Event Double Integration (EDI) model[1] by reformulating the intensity term $I\\_{i,0}$ in Eq. (5) using the EDI formulation:\n$$\n\\hat{I}\\_{i,0} = \\frac{(2n + 1) \\cdot B\\_i}{\\sum_{k=-n}^{n} \\exp\\left(C \\cdot \\sum\\_{z=0}^{k} E\\_{i,z}\\right)},\n$$\nwhere $B_i$ represents the blurred intensity, $E_{i,z}$ denotes the accumulated events, $C$ is the contrast threshold, and $n$ defines the temporal integration window of blur averaging.\n\nExperiments were performed on the Tanks and Temples dataset with a frame rate of 2FPS. Motion-blurred frames were synthesized adopting gamma correction and multi-frame averaging operations. We average every 30 frames to simulate blurring. We selected EvDeblurNeRF[2], currently the best-performing open-source event-based deblur scene reconstruction method, as our comparison.\n\n| Methods | Pose Estimation | PSNR↑ | SSIM↑   | $RPE_t$ ↓ | $RPE_r$ ↓ |\n|-|-|-|-|-|-|\n| EvDeblurNeRF  | CF-3DGS | 21.17 | 0.69  | 0.105 | 0.897 |\n| EvDeblurNeRF  | COLMAP | 22.58 | 0.70  | - | - |\n| EDI + EF-3DGS (Ours) | EDI + EF-3DGS (Ours) | **23.18** | **0.72**  | **0.062** | **0.642** |\n\nAs demonstrated in the table, our method achieves superior reconstruction quality to EvDeblurNeRF while simultaneously performing better pose estimation. Note that EvDeblurNeRF requires pre-computed COLMAP poses, which significantly restricts practical applicability.\n\n**Summary**: We appreciate the reviewer's concern about motion blur. We clarify that our method focuses on large pixel displacement challenges, which represents a complementary challenge to intra-frame motion blur. Our additional blur experiments suggest that our approach can be reasonably extended to handle motion blur scenarios. We acknowledge that more extensive realistic evaluation under various blur conditions would strengthen our claims, and we commit to conducting comprehensive validation with real-world motion blur and diverse challenging scenarios in the revised manuscript.\n\n> ### **Quality 2: Ground Truth Pose Concerns and Challenging Scenario Experiments**\n\n**Answer:** We sincerely thank the reviewer for this important suggestion. We would like to clarify the experimental setup and provide additional validation.\n\n**RealEv-DAVIS Ground Truth Pose Clarification:** In the SLOW scenario, COLMAP provides relatively accurate pose estimation due to sufficient visual overlap and features. The FAST scenario ground truth poses are then obtained by downsampling from these accurate SLOW scenario COLMAP poses, rather than directly running COLMAP on the sparse FAST frames. This enables fair quantitative comparison of pose estimation accuracy between methods. Our experiment evaluation protocol follows previous pose-free reconstruction methods, **focusing on comparisons with other pose-free approaches.**\n\n**Additional Challenging Scenario Experiments:** To further demonstrate event camera advantages, we constructed a FASTER scenario by selecting every 10th frame from the SLOW dataset and directly applied COLMAP pose estimation on these challenging sequences.\nWe added COLMAP+3DGS as a comparison baseline. Notably, in the corner and outdoor scenes, several frames exhibit registration failures where the estimated poses greatly deviate from the overall video trajectory, indicating COLMAP's limitations in challenging real-world scenarios.\n\n| Methods | SLOW | | FAST | |  FASTER | |\n|--|---|---|--|---|--|--|\n| | PSNR↑ | SSIM↑ | PSNR↑ | SSIM↑ |PSNR↑ | SSIM↑ |\n| CF-3DGS | 22.68 | 0.629 | 17.59 | 0.520 | 15.61 | 0.504 |\n| COLMAP+3DGS | 23.27 | 0.633 | 19.79 | 0.548 | 17.25 | 0.521 |\n| EF-3DGS (Ours) | **23.65** | **0.647** | **21.12** | **0.562** | **20.18** | **0.532** |\n\nThese results demonstrate that: (1) COLMAP+3DGS rendering quality degrades significantly as motion speed increase confirming pose estimation inaccuracies in challenging conditions. (2) Compared to CF-3DGS, our method maintains substantially better rendering quality in high-speed scenarios, highlighting our method robustness and demonstrating the potential of event cameras in high-speed scenarios.\n\nWe appreciate the reviewer's valuable suggestion and will include these challenging scenario comparisons in the revised manuscript to better highlight the effectiveness of our proposed event-aided 3DGS framework.\n\n> ### **Quality 3: Contrast Threshold Parameter Assumption**\n\n**Answer:** We appreciate the reviewer's concern about the contrast threshold assumption. To address this limitation, we propose a feasible approach that eliminates the dependency on the contrast threshold parameter through normalization, similar to scale-invariant processing in depth estimation.\n\nSpecifically, for $L_{EGM}$, we use the brightness change calculation method, letting $\\Delta \\hat{L} = \\hat{L}\\_t - L\\_{i,0}$, and $\\Delta L = \\sum\\_{i=1}^{n} E\\_{i,n}\\cdot C$, then formulate a normalized $L_{EGM}$,\n$$\nL\\_{EGM}= ||\\frac{\\Delta \\hat{L}}{||\\Delta \\hat{L}||\\_2} - \\frac{\\Delta L}{||\\Delta L||\\_2}||\\_2.\n$$\nFor $L_{LEGM}$, we apply similar normalization. We conducted experiments to validate this approach:\n\n| | PSNR↑ | SSIM↑   | $RPE_t$ ↓ | $RPE_r$ ↓ |\n|-|-|-|-|-|\n| normalization | 23.55 | 0.71  | 0.054 | 0.718 |\n| predifined $C$ (Ours) | **23.96** | **0.72**  | **0.049** | **0.626** |\n\nThe results show that while the normalization eliminates threshold dependency and maintains reasonable performance, the predefined threshold method achieves better overall reconstruction quality and pose accuracy. This suggests that an accurate pre-computed threshold provides superior optimization constraints for our framework.\n\n> ### **Question 1: IPWE vs IWE Computational Cost**\n\n**Answer:** We conducted experiments comparing our IPWE approach with the individual event warping method from SSL-E2VID[3] to quantify the computational efficiency improvement. The experimental results and training times are shown below:\n\n|  | PSNR↑ | SSIM↑   | $RPE_t$ ↓ | $RPE_r$ ↓  | Training Time↓|\n|-|--|--|-|----|---|\n| IWE | 23.82 | **0.72**  | **0.048** | **0.603** | 7.3h |\n| IPWE  (Ours) | **23.96** | **0.72**  | 0.049 | 0.626 |**2.5h** |\n\nOur IPWE approach achieves a significant reduction in training time while maintaining comparable reconstruction quality. The individual event warping (IWE) shows a significant computational overhead. This is primarily due to: (1) higher memory requirements for storing each event as (x,y,p,t) tuples, and (2) substantially increased per-iteration computation from warping individual event.\n\n> ### **Question 2 & 3 & 4: Some Clarifications**\n\n**Answer:** We appreciate the reviewer's questions and will clarify these points in the revised manuscript:\n\n**1) Training Stage Ratio:** Yes, the ratio refers to the proportion of training iterations between the two stages.\n\n**2) Ev-Baseline in Table:** Ev-Baseline refers to EvCF-3DGS. We renamed it to \"EvCF-3DGS\" for better method description clarity. This appears to be an oversight where we missed updating the table reference during writing.\n\n**3) Initial Camera Pose:** We follow a progressive optimization strategy where each new frame's pose is initialized using the pose from the previous frame. This provides a reasonable starting point based on the assumption of smooth camera motion between consecutive frames.\n\n[1] Pan, Liyuan, et al. \"Bringing a blurry frame alive at high frame-rate with an event camera.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019.\n\n[2] Cannici, Marco, and Davide Scaramuzza. \"Mitigating motion blur in neural radiance fields with events and frames.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.\n\n[3] Paredes-Vallés, Federico, and Guido CHE De Croon. \"Back to event basics: Self-supervised learning of image reconstruction for event cameras via photometric constancy.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021."}, {"review_id": "0QTliXDImV", "reviewer": "Reviewer_rd3s", "summary": "This paper, EF-3DGS: Event-Aided Free-Trajectory 3D Gaussian Splatting, proposes the first method combining event cameras with 3D Gaussian Splatting (3DGS) without requiring pre-estimated camera poses. The approach jointly optimizes camera poses and Gaussian positions using event data, while leveraging image data for color optimization. The method employs a two-stage training pipeline and is thoroughly validated through extensive experiments.", "questions": "### Questions about limitations\n**1. On the sharpness of pose optimization (related to L169):**  \nThe method appears to encourage maximizing alignment sharpness via event warping. However, event triggering is itself subject to latency and noise, especially under low-light conditions. Could this lead to overfitting to noise, i.e., aligning events that should not be aligned in the first place? If so, does this affect the robustness of the method under high noise or delayed event response?\n\n**2. On gradient-based loss (related to L189):**  \nEvent cameras operate asynchronously on a per-pixel basis. Does converting a series of events into a difference between two pixels rely on overly strict assumptions about the scene? Only assuming static scenes and consistent lighting seems not enough. For instance, if a light source is partially occluded while moving, or if surface materials exhibit strong anisotropy, could these factors lead to significant degradation in reconstruction quality?\n\n**3. On stability:**  \nEvent data is inherently noisy, especially under low lighting or with sensor latency. Moreover, 3D Gaussian Splatting with difference-based loss can be fragile . The proposed method appears not to include explicit noise-handling mechanisms. Given these factors, how does the approach achieve stable reconstruction? Were there any architectural or algorithmic choices that implicitly contribute to robustness?\n\n---\n\n\n### Other minor questions\n\n**4. Regarding the definition of $\\Delta L$ (L178):**  \nIt is unclear what the exact meaning of $\\Delta L$ is — is it the per-location brightness change? If so, the right-hand side of the formula might effectively reduce to the number of events. Could the authors clarify?\n\n**5. On dataset realism and evaluation settings (L249):**  \nThe use of COLMAP to obtain ground truth poses suggests the dataset is relatively ideal (i.e., low speed and good lighting). Are there any results — even qualitative — under high-speed or low-light conditions, where event cameras typically shine? Such evaluations would strengthen the claim that the method is robust and practically applicable.\n\n---\nDue to some remaining unresolved questions, my current recommendation is borderline accept. Nevertheless, I expect to raise my evaluation after reviewing the authors’ rebuttal.", "limitations": "Some important limitations are not adequately discussed. My concerns are already outlined in the \"Questions about limitations\" section of this review. I look forward to the authors’ reflections on these points.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- The paper is very well written, with clear logical flow and well-organized presentation, making it easy to follow and understand.\n- The theoretical formulation is clearly explained, and the authors properly acknowledge the inspiration and connections to prior work.\n- The experiments are thorough and comprehensive, covering ablation study.\n\nWeaknesses:\n- The discussion on the method's limitations may not be sufficiently comprehensive. The current design may restrict its applicability to certain environments.\n- The experimental setup lacks scenarios involving high-speed motion or low-light conditions, which are typically where event cameras provide the greatest advantage. This limits the demonstration of the full potential of combining event data with 3DGS.", "was_revised": true, "final_justification": "The authors’ rebuttal is very thorough and thoughtfully addresses most of my concerns. Combined with the fact that the original manuscript is of good quality, I have decided to raise my score accordingly.", "rebuttal": "We thank the reviewer for the constructive feedback and for recognizing our well-written paper, clearly explained formulation, and comprehensive experimental validation. We respond to each concern in detail. Please feel free to use the discussion period if you have any additional questions.\n\n> ### **Q1: Questions on the Sharpness of Pose Optimization**\n\n**Answer:** We understand the reviewer's concern regarding how event noise could lead to erroneous warping. We address this question from two perspectives:\n\n**Robustness Discussion:** Thank you for this insightful question. Indeed, the event-based vision community has extensively studied misalignment issues in event warping [1,2]. Misalignment primarily stems from two factors: 1) event noise [1] and 2) the ill-posed nature of the alignment optimization process (existence of local minima) [2], with these two factors often interfering with each other. While event noise inevitably causes alignment difficulties, we believe our method demonstrates certain noise robustness capabilities. In our approach, we derive the motion field from 3DGS-rendered depth and inter-frame poses, then use this motion field to warp events. This formulation involves optimizing two sets of parameters: 3DGS-rendered depth (corresponding to scene reconstruction) and camera poses (corresponding to camera trajectory estimation), creating a joint pose estimation and scene reconstruction problem (chicken-and-egg problem). Our method begins with monocular depth estimation networks to obtain a reasonable 3DGS initialization. With this initially good scene representation, the solution space is significantly reduced. Specifically, noise events typically lack spatial consistency, while our 3DGS depth-based motion field requires spatial geometric consistency, making our optimization inherently robust to isolated noise events. This constraint allows us to primarily optimize pose parameters (6DOF), making the maximization of IWE sharpness more robust, even in the presence of noise.\n\n**Noise Experiment Results:** To investigate the impact of noise on our method, we conduct experiment on Tanks and Temples and, following [2], randomly added noise events to the event stream based on a certain percentage of the total event count. The experimental results are shown below. Within a reasonable range (up to 30%), our method demonstrates considerable noise robustness, with pose estimation accuracy degrading gracefully rather than catastrophically.\n\n| Noise Level | PSNR↑ | SSIM↑   | $RPE_t$↓ | $RPE_r$↓  |\n|-|--|--|-|----|\n| 0% | 23.96 | 0.72  | 0.049 | 0.626 |\n| 10% | 23.28 | 0.71  | 0.064 | 0.661 |\n| 30% | 22.16 | 0.69  | 0.088 | 0.727 |\n| 50% | 20.24 | 0.65  | 0.125 | 0.854 |\n\n**Summary:** We share our thoughts on how noise affects event warping alignment and conducted preliminary noise experiments. While our primary contribution focuses on demonstrating robustness across different frame rates (high-speed scenarios), we acknowledge that comprehensive noise analysis requires further investigation in challenging real-world conditions. We find noise robustness to be a fascinating research direction and are willing to incorporate this discussion in the final version.\n\n> ### **Q2: Questions on the Gradient-Based Loss**\n\n**Answer:** We thank the reviewer for this insightful question. The gradient-based loss indeed relies on the brightness constancy assumption $L(x,y,t) = L(x+\\Delta x, y+\\Delta y, t+\\Delta t)$, which, combined with the small motion assumption, establishes the relationship between brightness changes, image gradients, and optical flow as shown in Eq. 11: $ \\Delta L(u) = - \\nabla L \\cdot \\dot{u}$. In typical scenarios, these assumptions generally hold over short time intervals thanks to the high temporal resolution characteristics of event cameras. The challenging scenarios mentioned by the reviewer (moving light occlusion and strong material anisotropy) do violate this assumption. However, this limitation is not unique to our event-based approach but represents a fundamental challenge shared across photometry-based reconstruction methods. As acknowledged in [3], existing event-based intensity reconstruction methods face the same constraint. Moreover, traditional 3DGS itself relies heavily on cross-view photometric consistency to establish geometric constraints—the scenarios described by the reviewer would equally compromise this consistency and degrade reconstruction quality in conventional 3DGS pipelines. Addressing such complex light-material interactions would require explicit modeling of surface material properties and illumination dynamics, which falls within the scope of inverse rendering rather than geometric reconstruction. We believe leveraging event data for inverse rendering represents an exciting future research direction that merits dedicated investigation.\n\n> ### **Q3: Questions on the Stability**\n\n**Answer:** Thank you for raising this important stability concern. For experiments evaluating our robustness, please refer to **Noise Experiment Results** in **Q1**. We would like to further explain our method's robustness:\n\n**1) Our Distinction from Other Event-Based NeRF or 3DGS Methods:** Previous event-based reconstruction methods EventNeRF and Event3DGS typically render frames at two different timestamps, compute the logarithmic brightness difference between these frames, and compare it with accumulated events between the two timestamps to calculate the loss. As the reviewer points out, computing differences between two images is inherently unstable, as differential operations further amplify noise and training instability. Moreover, events naturally record brightness changes rather than absolute intensities, making them unsuitable for rendering. Previous methods treat events as a **substitute for RGB images**, whereas we treat events as **auxiliary modal data to complement RGB images**. Through EGM and LEGM, we simultaneously leverage both brightness and brightness changes to assist RGB images in scene reconstruction, providing complementary supervision rather than replacement. Meanwhile, as camera motion speed increases (reflected by decreasing frame rates in our experiments), the potential of event data becomes increasingly prominent.\n\n**2) Robustness Potential of $L\\_{grad}$:** According to our derivation and Equation 9:\n$$IPWE\\_{i,j} = \\frac{1}{r+1}\\sum\\_{m=j-r}^{j}E\\_{i,m \\rightarrow j}$$\nFor each warped event $E\\_{i,m-j}$, we model it using a simple i.i.d. Gaussian noise model:\n$$\nE\\_{i,m-j} = \\frac{1}{C}\\Delta L + n\\_{m \\rightarrow j},\n$$\nwhere $n_{m \\rightarrow j} \\sim N(0, \\sigma^2)$ follows a Gaussian distribution. Therefore:\n$$IPWE\\_{i,j} = \\frac{1}{r+1}\\sum\\_{m=j-r}^{j}E\\_{i,m \\rightarrow j} = \\frac{1}{C}\\Delta L + \\frac{1}{r+1}\\sum\\_{m=j-r}^{j} n\\_{m \\rightarrow j}. $$\nThe variance of the noise term  $\\frac{1}{r+1}\\sum\\_{m=j-r}^{j} n\\_{m \\rightarrow j}$ is  $\\frac{\\sigma^2}{r+1}$, which demonstrates that averaging warped events significantly reduces noise variance by a factor of $(r+1)$. This averaging mechanism inherently provides noise suppression, making our $L_{\\text{grad}}$ more robust compared to methods that rely on single-frame differences. While this derivation may have some simplifications, we hope this example illustrates the robustness potential of $L_{\\text{grad}}$.\n\n> ### **Q4: Questions Regarding the Definition of $\\Delta L$**\n\n**Answer:** Yes, $\\Delta L$ represents the logarithmic brightness change. Each term $E\\_{i,m \\rightarrow j}$ on the right-hand side of the equation represents the accumulated events within the interval that are warped to the target timestamp $j$.\n\n> ### **Q5: Questions On Dataset Realism and Evaluation Settings**\n\n**Answer:** We thank the reviewer for this important question regarding dataset realism and challenging conditions evaluation. We would like to refer to our comprehensive response in **Quality 2: Ground Truth Pose Concerns and Challenging Scenario Experiments** from reviewer **UuWp's** reviews, where we have extensively addressed similar concerns about high-speed and challenging conditions.\n\nIn the **RealEv-DAVIS Ground Truth Pose Clarification** section, we explained our dataset's ground truth pose acquisition process which ensures fair quantitative comparison while simulating realistic challenging conditions.\n\nIn the **Additional Challenging Scenario Experiments** section, we conducted supplementary experiments under FASTER scenarios on RealEv-DAVIS where COLMAP frequently fails due to insufficient visual overlap and rapid motion. These experiments demonstrate that our event-aided method maintains superior performance compared to traditional approaches even under extreme challenging conditions.\n\nAs further elaborated in our response to reviewer **hsxP**'s **Justification for Pose-Free Approach** weakness, our novelty lies in effectively leveraging event camera properties to address two fundamental challenges that existing pose-free methods face in high-speed scenarios: (1) sparse scene reconstruction due to insufficient frame observations, and (2) inaccurate pose estimation caused by large inter-frame displacements.\n\n\n[1] Stoffregen, Timo, and Lindsay Kleeman. \"Event cameras, contrast maximization and reward functions: An analysis.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019.\n\n[2] Shiba, Shintaro, Yoshimitsu Aoki, and Guillermo Gallego. \"Secrets of event-based optical flow.\" European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022.\n\n[3] Zhang, Zelin, Anthony J. Yezzi, and Guillermo Gallego. \"Formulating event-based image reconstruction as a linear inverse problem with deep regularization using optical flow.\" IEEE Transactions on Pattern Analysis and Machine Intelligence 45.7 (2022): 8372-8389."}]

Author rebuttal:

Meta-review: This paper introduces EF-3DGS, a novel framework that integrates event cameras with 3D Gaussian Splatting for pose-free scene reconstruction. The method is well-motivated, clearly presented, and extensively validated across synthetic and real datasets, showing notable gains in both pose estimation and rendering quality, especially under high-speed scenarios. Reviewers praised the clarity, soundness of the pipeline, and thorough experiments, while raising concerns about the elegance of the two-stage optimization, assumptions in experimental settings (e.g., motion blur, thresholding), and the need for more discussion of limitations. The rebuttal was detailed, provided additional experiments (including blur robustness and challenging scenarios), and addressed most concerns satisfactorily. Overall, this is a solid contribution, and I recommend acceptance.
Decision: Accept (spotlight)