Title: EF-3DGS: Event-Aided Free-Trajectory 3D Gaussian Splatting
Venue: NeurIPS 2025
Accepted: yes
Keywords: 3D Gaussian Splatting (3DGS); Novel View Synthesis (NVS)
Abstract: Scene reconstruction from casually captured videos has wide real-world applications. Despite recent progress, existing methods relying on traditional cameras tend to fail in high-speed scenarios due to insufficient observations and inaccurate pose estimation. Event cameras, inspired by biological vision, record pixel-wise intensity changes asynchronously with high temporal resolution and low latency, providing valuable scene and motion information in blind inter-frame intervals. In this paper, we introduce the event cameras to aid scene construction from a casually captured video for the first time, and propose Event-Aided Free-Trajectory 3DGS, called EF-3DGS, which seamlessly integrates the advantages of event cameras into 3DGS through three key components. First, we leverage the Event Generation Model (EGM) to fuse events and frames, enabling continuous supervision between discrete frames. Second, we extract motion information through Contrast Maximization (CMax) of warped events, which calibrates camera poses and provides gradient-domain constraints for 3DGS. Third, to address the absence of color information in events, we combine photometric bundle adjustment (PBA) with a Fixed-GS training strategy that separates structure and color optimization, effectively ensuring color consistency across different views. We evaluate our method on the public Tanks and Temples benchmark and a newly collected real-world dataset, RealEv-DAVIS. Our method achieves up to 3dB higher PSNR and 40% lower Absolute Trajectory Error (ATE) compared to state-of-the-art methods under challenging high-speed scenarios.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper, EF-3DGS: Event-Aided Free-Trajectory 3D Gaussian Splatting, proposes the first method combining event cameras with 3D Gaussian Splatting (3DGS) without requiring pre-estimated camera poses. The approach jointly optimizes camera poses and Gaussian positions using event data, while leveraging image data for color optimization. The method employs a two-stage training pipeline and is thoroughly validated through extensive experiments.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The paper is very well written, with clear logical flow and well-organized presentation, making it easy to follow and understand.\n- The theoretical formulation is clearly explained, and the authors properly acknowledge the inspiration and connections to prior work.\n- The experiments are thorough and comprehensive, covering ablation study.\n\nWeaknesses:\n- The discussion on the method's limitations may not be sufficiently comprehensive. The current design may restrict its applicability to certain environments.\n- The experimental setup lacks scenarios involving high-speed motion or low-light conditions, which are typically where event cameras provide the greatest advantage. This limits the demonstration of the full potential of combining event data with 3DGS.\n\nQUESTIONS:\n### Questions about limitations\n**1. On the sharpness of pose optimization (related to L169):**  \nThe method appears to encourage maximizing alignment sharpness via event warping. However, event triggering is itself subject to latency and noise, especially under low-light conditions. Could this lead to overfitting to noise, i.e., aligning events that should not be aligned in the first place? If so, does this affect the robustness of the method under high noise or delayed event response?\n\n**2. On gradient-based loss (related to L189):**  \nEvent cameras operate asynchronously on a per-pixel basis. Does converting a series of events into a difference between two pixels rely on overly strict assumptions about the scene? Only assuming static scenes and consistent lighting seems not enough. For instance, if a light source is partially occluded while moving, or if surface materials exhibit strong anisotropy, could these factors lead to significant degradation in reconstruction quality?\n\n**3. On stability:**  \nEvent data is inherently noisy, especially under low lighting or with sensor latency. Moreover, 3D Gaussian Splatting with difference-based loss can be fragile . The proposed method appears not to include explicit noise-handling mechanisms. Given these factors, how does the approach achieve stable reconstruction? Were there any architectural or algorithmic choices that implicitly contribute to robustness?\n\n---\n\n\n### Other minor questions\n\n**4. Regarding the definition of $\\Delta L$ (L178):**  \nIt is unclear what the exact meaning of $\\Delta L$ is — is it the per-location brightness change? If so, the right-hand side of the formula might effectively reduce to the number of events. Could the authors clarify?\n\n**5. On dataset realism and evaluation settings (L249):**  \nThe use of COLMAP to obtain ground truth poses suggests the dataset is relatively ideal (i.e., low speed and good lighting). Are there any results — even qualitative — under high-speed or low-light conditions, where event cameras typically shine? Such evaluations would strengthen the claim that the method is robust and practically applicable.\n\n---\nDue to some remaining unresolved questions, my current recommendation is borderline accept. Nevertheless, I expect to raise my evaluation after reviewing the authors’ rebuttal.\n\nLIMITATIONS:\nSome important limitations are not adequately discussed. My concerns are already outlined in the \"Questions about limitations\" section of this review. I look forward to the authors’ reflections on these points.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-15"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a novel method for 3D Gaussian Splatting of static scenes using images and event streams as input, while simultaneously optimizing the camera trajectory. The input images are temporally sparse, whereas the events provide dense temporal information. The proposed method consists of two stages. In the first stage, the parameters of the 3D Gaussians and the camera trajectory are jointly optimized using a loss function derived from the Event Generation Model (EGM), Contrast Maximization (CMax), and Photometric Bundle Adjustment (PBA). Since events do not contain information about absolute brightness values, accurate color reconstruction is not feasible in this stage. Therefore, in the second stage, only the color of the 3D Gaussians are refined using the RGB images.\n\nIn the experiments, the authors used the publicly available Tanks and Temples dataset, as well as the RealEv-DAVIS dataset constructed by the authors, to quantitatively and qualitatively evaluate the effectiveness of the proposed method. Additionally, the authors assessed the robustness of the proposed method to noise in the initial camera poses and demonstrated that it maintains strong performance under high noise levels compared to existing approaches.\n\nSTRENGTHS AND WEAKNESSES:\n**Quality**\n1. Questions remain regarding the experimental assumptions. The authors simulate large inter-frame displacements by reducing the number of frames. However, high-speed camera motions would typically result in motion blur in the captured images. The experiments in the paper assume that sharp, blur-free images are provided as input, which may not accurately reflect real-world conditions.\n2. In the RealEv-DAVIS dataset, COLMAP is used to obtain ground-truth camera poses, which implies that accurate camera pose can be estimated from images. To more clearly demonstrate the advantages of event cameras, it may be beneficial to conduct additional experiments where pose estimation from images is challenging or unreliable.\n3. Although the experiments in this paper assume that the threshold $C$ at which events are triggered is known, this assumption may not hold in general.\n\n**Clarity**\n1. The process of the proposed method is very clearly explained in the paper and is easily understood.\n2. The authors present extensive quantitative and qualitative experimental results, which effectively demonstrate the efficacy of the proposed method and aid the reader's understanding.\n\n**Significance**\n1. This paper serves as an example demonstrating the advantages of event cameras over frame-based cameras in 3D Gaussian Splatting. However, as previously noted, questions remain regarding the experimental setup, and the full potential of event cameras has not been comprehensively evaluated.\n\n**Originality**\n1. The authors leverage the unique characteristics of events by integrating frame and image information through the EGM, and by utilizing motion information encoded in the events for optimization via Contrast Maximization.\n\nQUESTIONS:\n1. Contrast Maximization optimization is generally computationally expensive. In this paper, instead of computing the Image of Warped Events (IWE) for each individual event, the authors propose computing the Image of Piece-wise Warped Events (IPWE) by warping events that occur within a predefined time interval collectively. How much does this modification reduce the overall optimization time?\n2. In Section 4.4, the authors state, “The ratio between the first and second stages is empirically set to 4:1.” What does this ratio refers to? Does it indicate the ratio of the number of iterations used in the optimization for each stage?\n3. What is Ev-Baseline in Table 8?\n4. How is the initial camera poses given?\n\nLIMITATIONS:\nyes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-25"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces the event cameras to aid scene construction from a casually captured video for the first time, and propose Event-Aided Free-Trajectory 3DGS, called EF-3DGS, which seamlessly integrates the advantages of event cameras into 3DGS through three key components\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\nClarity and Visualizations: The paper is exceptionally well-written, and the figures are clear and easy to understand.\n\nRich Content: The overall content presented in the paper is comprehensive and substantial.\n\nWeaknesses: \n\n1. Necessity of Two-Stage Optimization: The paper proposes a two-stage framework: first optimizing the shape of Gaussian spheres for texture features, and then optimizing colors. While this idea is novel, its necessity is debatable. Specifically, in Section 4.1 (EGM optimization), could the optimization directly use the given reference image plus event data? For example, by calculating the difference between the 3DGS-rendered image and the reference image, and then using this difference with event data as a loss function. Combined with the authors' introduced optical flow warping, could this directly address the color issue? I would be very interested in seeing an ablation study on this specific aspect.\n\n2. \"Pose-Free\" Claim Discrepancy: The paper's title claims a \"pose-free\" approach, yet the methodology section lacks any explicit description or discussion of how this pose-free aspect is achieved. Is this work directly built upon an existing pose-free 3DGS method, or is there a novel contribution to pose estimation that is not clearly articulated?\n\n3. Justification for Pose-Free Approach: The necessity of a pose-free method needs stronger emphasis. A detailed discussion on how inaccuracies in pose estimation in real-world scenarios significantly impact other methods would strengthen the paper's claims and highlight the importance of their \"pose-free\" contribution.\n\nQUESTIONS:\nPlease see the weakness.\n\nLIMITATIONS:\nSee the weakness.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposed EF-3DGS. It is a 3DGS model based on 1. high-frequency event camera stream 2. RGB video frames and 3. no trajectories aka camera poses. The overall problem setup is compelling since event camera captures a continuous stream that encapsulate fine-grained motion and RGB video frames contains rich visual details for constraining the overall structure. Such a setup without any presumption on trajectory or poses is worth studying. This paper integrates the principle of EGM, CMax and LEGM into this setup. Further, the proposed PBA refines the continuous trajectory supervised by sparse video frames. The overall pipeline is split into two stage where the first stage optimize the overall geometry and monochrome color and second stage refines the color with RGB video stream. Results show effectiveness on both reconstruction quality and pose/trajectory estimation.\n\nSTRENGTHS AND WEAKNESSES:\nStrength\n\n- The  proposed pipeline which integrates different modalities, 3DGS, event generation model and bundle adjustment, is overall sound and reasonable.\n- Sufficient experiments that both presents results quantitatively and qualitatively, with visualizations and numerical comparisons. Results show good performance on both reconstruction and trajectory recovery.\n\nweakness\n\n- Figure 3 might be too small can hardly tell much from the comparison (but the one in the appendix is ok so not a big issue). The location of figure 3 need adjust as well.\n- With the loss mentioned in eq.18, both rgb and grayscale loss are involved. It is unclear how they are optimized together.\n\nMethodology part need some reorganization:\n\n- on page 5 where IPWE is introduced, it is illustrated in Figure 6 but Figure 6 is on page 9.\n- in 4.4, the argmin equation is minimizing $\\mathcal{L}_{event}$ but this loss is not introduced until 4.5.\n\nQUESTIONS:\nSuch setup is quite commonly observed on robotics and autonomous vehicles. I would like to know you thoughts on how far away from EF-3DGS to real-time, online processing, since inherently your work does not need any trajectory. Is it straight forward or still unanswered questions?\n\nminor suggestions:\n\n- place figure 3 and figure 6 in a better position, somewhere closer to it is first mentioned.\n- some basic introduction around EGM, CMax and LEGM might go to the preliminary sections. Otherwise it is confusing which part in the methodology is the contribution of this work.\n- It can also be interesting to visualize the optimization process, that as loss being optimized, the warped image gets sharper and trajectory is optimized simultaneously.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}