Event3DGS: Event-Based 3D Gaussian Splatting for High-Speed Robot Egomotion

Authors: tianyi  xiong
jiayi  wu
botao  he
cornelia  fermuller
yiannis  aloimonos
heng  huang
christopher a metzler
Venue: robot-learning_org_CoRL_2024_Conference
Published: 2024-05-31 05:52:59

Abstract: By combining differentiable rendering with explicit point-based scene representations, 3D Gaussian Splatting (3DGS) has demonstrated breakthrough 3D reconstruction capabilities. 
    However, to date 3DGS has had limited impact on robotics, where high-speed egomotion is pervasive: Egomotion introduces motion blur and leads to artifacts in existing frame-based 3DGS reconstruction methods. 
    To address this challenge, we introduce Event3DGS, an event-based 3DGS framework.
    By exploiting the exceptional temporal resolution of event cameras, Event3GDS can reconstruct high-fidelity 3D structure and appearance under high-speed egomotion. 
    Extensive experiments on multiple synthetic and real-world datasets demonstrate the superiority of Event3DGS compared with existing event-based dense 3D scene reconstruction frameworks; Event3DGS substantially improves reconstruction quality (+3dB) while reducing computational costs by 95\%. 
    Our framework also allows one to incorporate a few motion-blurred frame-based measurements into the reconstruction process to further improve appearance fidelity without loss of structural accuracy.

Main paper: 
 novel view synthesis and 3d gaussian splattings: 
 3D scene reconstruction and novel-view synthesis is a fundamental task in graphics and computer vision [16, 17, 18, 19] , boosting applications in autonomous driving [20] , robotics [21, 22] and virtual reality [23] . NeRF [1] and its variants [24, 25, 26, 27, 28, 29] model a scene implicitly with a MLP-based neural network and utilize differentiable volume rendering, achieving near photorealistic renderings with high fidelity and fine details. However, since a large number of points need to be sampled to accumulate the color of each pixel, these methods suffer from low rendering efficiency and long training time. Extended works on radiance field aims to accelerate the pipeline by interpolating values from explicit density representations such as points [30] , voxel grids [31, 32, 33] , or hash grids [34] . Although these methods achieve higher efficiency than the vanilla MLP version, they still need multiple queries for each pixel, lacking real-time rendering capacity.In light of these challenges, recent research has explored alternative 3D representations for better efficiency and visual fidelity. 3D Gaussian Splatting (3DGS) [2] employs a set of optimized Gaussian splats to achieve state-of-the-art reconstruction quality and rendering speed. Initialized from sparse SfM [35, 36, 37] point clouds, 3DGS is trained via differentiable rendering to adaptively control the density and refine the shape and appearance parameters. A tiled-based rasterizer is proposed to allow for real-time rendering. Multiple works have applied the technique in applications such as SLAM [38, 39] , dynamic reconstruction [40, 41] , and scene editing [42, 43] . However, all these methods require clear RGB images as input.Event-based and event-aided 3D reconstruction [44, 45, 46, 47, 48, 49, 50, 51, 52] and radiance field rendering [53, 9, 54, 55, 56, 57, 13] represent a paradigm shift in computer vision and graphics, enhancing the perception of dynamic scenes with high temporal resolution and accuracy. Weikersdorfer et al. [58] demonstrated event-based stereo reconstruction, illustrating the potential for reconstructing 3D scenes using data from stereo event cameras. However, stereo matching can be challenging due to the sparse nature of event camera data, which often leads to unstable performance in depth estimation [59] . Muglikar et al. [45] enhanced depth sensing by integrating an event camera with a laser projector. While this approach achieves better depth accuracy, the inclusion of a laser projector complicates its effectiveness in outdoor environments with challenging illumination conditions. Previous works introducing event-based radiance fields include Ev-NeRF [53] , EventNeRF [9] , and E-NeRF [60] . These approaches leverage the inherent multi-view consistency of NeRFs [1] , providing a strong self-supervision signal for extracting coherent scene structures from raw event data. However, they inherit NeRF's high computational complexity and challenges in real-time rendering. NeRF's implicit representation complicates editing and integration with traditional 3D graphics processing pipelines.Our proposed Event3DGS offers explicit, interpretable scene geometry depiction and editable highfidelity 3D radiance field reconstruction. It allows seamless integration with established graphics pipelines and enables streamlined optimization. Event3DGS is robust under high-speed egomotion, low light, and high dynamic range scenarios where RGB cameras fail to deliver. By combining the event camera's hardware advantages with 3DGS's efficient rendering, our pipeline enables realtime 3D rendering of diverse scenes with low latency, low data bandwidth, and ultra-low power consumption, which supports 3D mapping at a higher operating speed. We first utilize a neutralization-aware accumulator (for mitigating the cancellation of positive and negative events) and sparsity-aware sampling strategy (for reconstruction in non-event regions) to process the input event stream into frames. Then, the sampled event frames are utilized as differential supervision between the corresponding rendered views, optimizing the 3D Gaussians to reconstruct sharp structures and apperance from fast egomotion. We train Event3DGS progressively to better represent geometric details. As an optional component, we integrate a few motion-blurred RGB images from an attached frame-based camera into the pipeline. By embedding motion blur formation into the rasterizer and employing a parameter-separable refinement strategy, we calibrate the colorization while preserving structural details.The proposed Event3DGS aims to efficiently reconstruct a 3D scene representation from a given sequence of events (either grayscale or color) under high-speed robot egomotion and low-light conditions. Fig. 2 illustrates the overall architecture. Unlike image-based reconstruction, our Event3DGS approach does not directly supervise the absolute radiance of rendered images during optimization. Instead, we integrate the event formation process into the 3DGS pipeline and utilize the observed events as ground truth to implement differential self-supervision within the gradient-based optimization framework. We propose progressive training to further boost reconstruction for fine-grained structural details. Optionally, to solve the scale ambiguity problem of radiance inherent in event data, we describe parameter separable refinement approach, aligning geometrically sharp Event3DGS with true scene radiance and texture details using a small number of blurred views. 
preliminary: 
 3D Gaussian Splatting (3DGS) [2] explicitly represents a scene with a set of anisotropic 3D Gaussians (ellipsoids). Each Gaussian is defined by a 3D covariance matrix Σ with its center point µ:EQUATIONTo preserve the valid positive semi-definite property during optimization, the covariance matrix is decomposed into Σ = RSS T R T , where S ∈ R 3 + represents scaling factors and R ∈ SO(3) is the rotation matrix. Each Gaussian is also described with an opacity factor σ ∈ R, and spherical harmonics C ∈ R k for modeling view-dependent effects.During optimization, 3D Gaussian Splatting adaptively controls Gaussian density via densification in areas with large view-space positional gradients and pruning points with low opacity. For rendering, the 3D Gaussians G(x) are first projected onto the 2D imaging plane G ′ (x), then a tile-based rasterizer is applied to enable fast sorting and α-blending. The color of pixel u is calculated via blending N ordered overlapping points:EQUATIONwherec i = f (C i )is the color modeled via spherical harmonics, andα i = σ i G ′ i (u)is the multiplication of opacity and the transformed 2D Gaussian.The input to our Event3DGS pipeline comprises a continuous stream of events e = (t, u, p), each indicating a detected increase or decrease in logarithmic brightness (denoted by the polarity p ∈ (-1, 1)) at a specific time instant t and pixel location u = (x, y). In order to efficiently utilize event data, a common practice is to use event windows to accumulate corresponding events, which requires us to slice the event stream. In event-based 3D radiance field reconstruction pipelines, the slicing strategy of the event stream affects the scene's reconstruction quality. This impact is particularly notable within our pipeline, as neutralization is inevitable during the accumulation of polarity. Existing works [9, 13] have shown that using constant short windows leads to poor propagation of high-level illumination, and using constant long windows often leads to poor local detail. To mitigate the information loss, we design a neutralization-aware event slicing strategy. Our slicing strategy considers the number of events and the neutralization moment to sample the length of the event integration window adaptively by (1) performing slicing when the number of events reaches the threshold, (2) performing slicing where neutralization occurs on many pixels (set threshold manually). This not only ensures the diversity of window lengths but also minimizes the loss of detailed information caused by neutralization.Uniform radiance regions typically do not trigger events, resulting in spatial sparsity of event data as supervision signals. To mitigate this issue, we introduce low-level Gaussian noise N (µ noevt , σ 2 noevt ) during the sampling process to augment pixels with no events throughout the entire event window, which enhances the gradient-based optimization on uniform radiance regions and makes our pipeline more robust to noise events. This is expressed formally in Eq. 3:EQUATIONwhere E u denotes the accumulation of all event polarities triggered at pixel coordinate u within the current event window, ∆ is the fixed event threshold, σ noevt = 0.2 in our experiments, t s and t e are the timestamps of the window start and the window end, respectively.Event data with high temporal resolution provides supervision signals with sharp structural information, allowing 3D Gaussian Splatting (3DGS) to perform fine-grained reconstruction of scene structure under high-speed egomotion. The multi-view consistency of event sequences guarantees the learnable Gaussians to continuously converge to the ground truth geometric structure and logarithmic color field of the scene during optimization. Our event rendering loss L event (t s , t e ) compares the recorded events with the differential signal generated by corresponding view renderings according to the event formation model. Following [2] , it primarily comprises two components: the L 1 loss, which measures the absolute log-radiance change difference at each pixel, and the structural dissimilarity loss L DSSIM [61] , which accounts for the structural information calculated by neighboring pixels. We define them as follows:EQUATIONEQUATIONwhere C(t) denotes the 2D rendering under the view at time t, g is a gamma correction value initialized to 2.2 in our experiments, E represents the accumulation of all event polarities triggered within the field of view (FOV), F is the RGGB Bayer filter [9] , which only is applied for color events. The total loss can be written as shown in eq. ( 6), and we set λ DSSIM to 0.2 in our experiments.EQUATIONThe point cloud initialization significantly affects the reconstruction quality of Gaussian Splatting [2, 62] . With precise initial positions, finer structural details can be captured via densification and division of Gaussian splats during training. While Structure-from-Motion (SfM) methods provide accurate point initializations for conventional RGB-based 3D Gaussian Splatting, obtaining precise sparse point initializations directly from event streams is challenging due to the absence of a sufficiently accurate event-based SfM pipeline. Alternatively, we have discovered that Event3DGS, when trained from a random initialization, can itself serve as a relatively accurate initialization. Consequently, we propose a progressive training approach to progressively capture geometric details in under-reconstructed areas. Specifically, given a pretrained Event3DGS that originated from random initialization, we apply an opacity threshold α pro to select Gaussian splats with high opacity and use their center positions as the initialization for the subsequent training rounds. A further detailed illustration is provided in Appendix F.Although severely motion-blurred RGB images are challenging for radiance field training due to structural degradation, their true radiance scale and texture information complement event data. We aim to refine the appearance of Event3DGS via training on a small amount of motion blurred inputs, to improve visual fidelity while maintaining sharp scene structure.In the realm of physics, camera motion blur stems from the amalgamation of radiance induced by the camera's movement. According to the physical image formation, camera motion blur is produced by the integration of radiance during camera movement, which can be mathematically represented with the following equation:EQUATIONwhere I blur represents the blurry image, I(P τ ) is the latent sharp image captured at camera pose P τ ∈ SE(3). To simplify the integral calculation, we approximate it as a finite sum of N radiance values I(P τi ), where τ i are the midpoint timestamps of a finite number of event integration windows (EIW) within the exposure interval (from τ s to τ e ). To incorporate motion effects due to camera movement during frame capturing into the differentiable rasterization process, we incorporate the above physical formation process of motion blur into the rendering equation:EQUATIONwhere C blur denotes the rendered color at pixel (x,y) given by blur-aware rasterization, G are the 3D Gaussian model parameters, N EIW represents the number of event integration windows within the exposure interval. The loss function L blur can be written as:EQUATIONTo improve the fidelity of scene appearance via a few blurry RGB images while preserving sharp structural details from event sequences, we separate the learnable parameters into two groups. The structure-related parameters include the position µ, scaling factor S, and rotation factor R; the appearance-related parameters include opacity α and spherical harmonics (SH) coefficients. When trained on event streams, all parameters of Event3DGS are optimized to learn the structure and the approximate logarithmic color field of the target scene. After the parameters have converged on the event stream, we fix the structure-related parameters and only calculate gradients on the appearancerelated parameters, using blurry RGB images as supervision to refine the scene's appearance. We scale the learning rate of opacity α by η α = 0.05 to inhibit drastic changes in density. 
experiments: 
 Synthetic and Real-world Datasets We evaluate our method using both synthetic and real data.For synthetic scenes, we adopt the dataset proposed in [9] , which generates 7 sequences with 360 • camera rotations around each 3D object, simulating event streams from 1000 views. For real-world scenes, we first capture videos from a fast-moving RGB camera, then extract frames and estimate camera parameters by COLMAP [35] . We utilize v2e [63] with bayes filter [9] to emulate colorful event streams. The emulated sequences cover both indoor and outdoor scenes under various illumination conditions. We also use the experimental event sequences from [9] , which are captured with a DAVIS-346C color event camera on a spinning table illuminated by a 5W light source.We report three popular metrics to evaluate our methods: Peak Signal-to-Noise Ratio (PSNR) [64] , Structural Similarity Index Measure (SSIM) [61] , and AlexNet-based Learned Perceptual Patch Similarity (LPIPS) [65] . Following [9] , we apply a linear transformation in the logarithmic space for all our and baseline results. Our implementation is based on the official 3DGS [2] framework. We train our model on a single NVIDIA RTX 6000Ada GPU for 30k iterations and filter the Gaussians with opacity threshold α ≥ 0.9 for progressive training. We randomly initialize the point cloud according to the scale of each training scene and set the other hyperparameters and optimizer as default.Baselines We benchmark our work against a NeRF-based method, EventNeRF [9] , and a naive baseline E2VID [66] + NeRF [1] , which cascades the event-to-video pipeline E2VID to a vanilla 3D Gaussian Splatting. For synthetic and low-light scenes, we directly render RGB and depth images from the official checkpoints of EventNeRF and only reproduce their training for efficiency evaluation. For real-world scenes, we train EventNeRF for 500k iterations using their default settings. Additional comparisons with deblurring baselines are included in Appendix B.Scene E2VID [66] + 3DGS [2] EventNeRF [9] Event3DGS (event-only) Event3DGS excels in reconstructing sharp structures and appearance details.PSNR ↑ SSIM ↑ LPIPS ↓ PSNR ↑ SSIM ↑ LPIPS ↓ PSNR ↑ SSIM ↑Synthetic Scenes As demonstrated in Tab. 1, Event3DGS consistently outperforms both baselines across all synthetic scenes in all metrics. On average, our method achieves a +2.61dB higher PSNR, a 2.15% higher SSIM, and a 50% lower LPIPS.Real-world Scenes Given that the E2VID [66] + 3DGS [2] baseline performs poorly on forwardlooking real-world scenes, we compare our method only with EventNeRF [9] . As shown in Tab. 2, Event3DGS significantly outperforms EventNeRF [9] across all real scenes and metrics, achieving +3.41 dB higher PSNR, 43.9% higher SSIM, and 65.2% lower LPIPS on average.We visualize depth maps and renderings on 3 synthetic scenes and 3 real-world scenes. Fig. 4 shows that our method preserves sharper, more consistent structures and cleaner backgrounds compared to EventNeRF [9] . Event3DGS is able to capture detailed information of object edges and geometric discontinuities, such as ficus leaves (2 nd row), drum racks (3 rd row) and shoelaces (6 th row). Our renderings also exhibit higher contrast and sharper details, particularly in highlights and reflections.In the soccer shoe scene, our method captures the reflected lights and corresponding depth, while EventNeRF [9] fails to reconstruct these details. In the bike sample, EventNeRF fails to represent high-frequency details of the grass, whereas our method accurately reconstructs the grass geometry and preserves details in the background. Event3DGS also demonstrates robustness in low-light conditions. As shown in Fig. 3 , our method learns sharper object details (e.g. edges of leaves) with fewer noisy artifacts. We include additional visualization results and deployment on quadrotor in Appendix D and Appendix A respectively.Progressive Training Fig. Blur-aware Appearance Refinement We adaptively fine-tune appearance-related parameters with 50 -300 iterations for each synthetic scene and plot the average PSNR in Fig. 6 . As shown, using up to 10 blurry RGB images already yields a noticeable enhancement in rendering quality. We provide more comprehensive ablation studies in Appendix E.Model Efficiency. As shown in Tab. 3, Event3DGS reduces the training time of EventNeRF from 9 hours to less than 20 minutes and achieve over 2000× higher FPS, enabling real-time rendering.  
conclusion: 
 Event cameras are a promising tool for sensing and navigating with high-speed robotics. Today, inverse differentiable rendering methods, like EventNeRF [9] , are the most effective approach to turn event streams into dense 3D reconstructions. Unfortunately, the computational cost of these methods-hours per scene-make them impractical for most applications. Benefiting from the efficiency of 3D Gaussian Splatting, we present Event3DGS, an event-based 3D dense reconstruction method that achieves state-of-the-art reconstruction quality and significantly accelerate training and rendering. By integrating differential event supervision, sampling, progressive training strategies tailored to event data characteristics, Event3DGS achieves high-fidelity 3D reconstruction under high-speed egomotion and low-light scenarios. Optionally, we introduce parameter-separable finetuning to further improve appearance fidelity with a few motion-blurred RGB images, with negligible computational overhead.This work makes a substantial step towards real-time dense 3D reconstruction with events. By extending the 3D Gaussian Splatting framework to perform reconstruction with event data, our work enables event-based dense 3D reconstructions at a rate 20× faster than existing methods. Still, there remains substantial room for improvement and our method is far from real-time. Further reducing run-times to enable real-time dense 3D reconstruction from events represents an important and exciting direction for future research. 
events rendering depth: 
 Blurry Image Rendering DepthAccurately reconstructing the structure and appearance of 3D scenes from a sequence of 2D images is a fundamental problem in robotics. By combining differentiable rendering models with continuous 3D scene representations, recent "inverse differentiable rendering" (IDR) methods (e.g., Neural Radiance Fields (NeRF) [1] and 3D Gaussian Splatting (3DGS) [2] ) have made significant strides in addressing this challenge. Given a sequence of high-quality 2D images, these methods can accurately reconstruct dense 3D geometry and provide near photo-realistic renderings from new views.However, the accuracy of these methods is fundamentally limited by the quality of their input images: For example, motion blur can severely hamper IDR-based methods ability to reconstruct 3D geometry. Unfortunately, egomotion-induced motion blur is pervasive in the images captured by real-world robotics systems (e.g., fast-moving drones). Although motion-blur-aware IDR methods have sought to mitigate these effects [3, 4, 5, 6, 7, 8] , severe motion blur still fundamentally limits the quality of 3D frame-based reconstructions.Recent works have sought to overcome these limitations by combining neural radiance fields with event cameras [9, 10, 11, 12, 13, 14] . Event cameras are a novel sensing technology that offers several advantages over frame-based cameras, particularly in the presence of high-speed robot egomotion. By asynchronously recording pixel-level changes in log-intensity, event cameras provide microsecond-level temporal resolution, are robust to motion blur, and have a much higher dynamic range than conventional frame-based cameras [15] . As a result, methods combining neural radiance fields with event data can reconstruct scenes from measurements with substantial egomotion. However, existing methods are impractically slow (hours per reconstruction) and, as we will demonstrate, offer substantial room for improvement with respect to reconstruction accuracy.In this work we introduce Event3DGS, an event-based 3D reconstruction framework built upon 3D Gaussian Splatting. By integrating the event formation process and differential supervision into the 3DGS framework, Event3DGS recovers multi-view consistent scene representation by minimizing the approximate difference between the integral of observed events and the radiance variations across different rendering views. We also introduce a novel sampling and progressive training strategies to accommodate the sparse characteristics inherent in event data. In addition, Event3DGS can exploit a small number of blurred frame-based images for additional appearance refinement.Extensive experiments on both simulation and real-world datasets demonstrate that compared to existing event-based IDR methods, Event3DGS can generate comparable or better reconstructions (see Fig. 1 ) of appearance and geometry while substantially reducing computational costs. Our contributions can be summarized as follows:1. We introduce a 3D Gaussian Splatting framework for reconstructing appearance and geometry solely from event data. 2. We propose a sparsity-aware sampling and progressive training approaches tailored to event data that improves reconstruction accuracy. 3. We incorporate motion blur into our reconstruction formation process and enable our framework to optionally use motion-blurred frame-based RGB images to improve reconstruction quality.2 Background and Related Work 
 
 Appendix: DeblurGS [4] Ours (event-only) GT Figure 8 : Qualitative comparison with deblurring baselines on synthetic dataset. Data was generated using blender and event simulator [9] . We only report the scenes where rendering of DeblurGS [4] can align with the test views. Event3DGS demonstrates more accurate structural details and better multi-view consistency than baseline methods.results in Tab. 5 and Fig. 9 respectively. Although DeblurGS roughly deblurs the input images and achieves higher reconstruction quality than the vanilla 3DGS, it fails to preserve multi-view consistency due to the existence of motion blur, causing under-representation in structural details (e.g. bicycle spokes, keyboard, edges of leaves, shoelaces in Fig. 9 ). As shown in Tab. 5, Event3DGS clearly outperforms baseline methods by an average of +0.44dB higher PSNR, 19% higher SSIM and 33% lower LPIPS. Table 5 : Quantitative comparison with deblurring baselines on real-world dataset. Due to the inherent radiance scale ambiguity of event data and the absence of direct color-wise supervision, Event3DGS does not achieve superior PSNR across all scenes. However, it demonstrates the highest structural and perceptual accuracy.Notably, DeblurGS [8] requires an average of 3.5 hours for training on a synthetic scene due to the high computational cost of motion-blur formation and long training rounds. Event3DGS converges in just 18 minutes with the same hardware (a single NVIDIA RTX 6000Ada GPU), demonstrating significantly higher efficiency.DeblurGS [4] Ours (event-only) GT EventNeRF [9] Ours (Event-only)Figure 10 : Visualization results on low-light scenes. Data was experimentally captured using DAVIS-346C [9] . We randomly select two rendered views for each scene. For EventNeRF [9] , we directly render images from their official checkpoints.  


Decision: Summary: This paper proposes Event3DGS which  combines differentiable rendering with event cameras to enhance 3D reconstruction under high-speed egomotion, significantly improving quality and efficiency compared to existing methods.

Strengths:
* Shows superior performance over state-of-the-art approaches, both quantitatively and qualitatively.
* Significantly faster training and inference due to the use of 3D Gaussian Splatting (3DGS).
* Incorporates a method to handle motion blur, enhancing appearance fidelity without compromising structural accuracy.

Weakness:
* Key sections, such as progressive training, lack detailed explanation.
* Insufficient comparison to baseline methods like EventNerf; needs more detailed analysis to explain why the proposed method works better.
* No ablation study on the impact of individual components.
* Experiments do not directly connect to real-world robotics scenarios, and the assumption of known camera poses is a limitation in high-speed ego-motion settings.
* This paper fits more for a computer vision conference paper than CoRL.

----

Post rebuttal: The reviewers are happy with the authors' replies and have reached a consensus on accepting the paper.

Reviews: review: title: title: Significant step in performance, writing is lacking.
paper_summary: paper_summary: This paper proposes a new method for leveraging event-based cameras, or DVS (dynamic vision sensor) cameras, to enable 3D reconstruction. While prior works used NeRF representations for a similar task, this work proposes to leverage Gaussian splats. The contributions are: neutralization-aware slicing, sparsity-aware sampling, blur-aware rasterization and separable apperance refinement. The results show a significant improvement in quality compared to related baselines.
main_review: main_review: This paper addresses an interesting problem: 3D reconstruction using DVS cameras, and optionally RGB frames. This is a meaningful problem in robotics as fast ego-motion can make 3D reconstruction and SfM pipelines fail. Another application not mentioned by the authors is high-speed, dynamic manipulation, which is particularly relevant in industrial settings. Event-based cameras are one possible solution. 

The results are good and show Event3DGS outperforms the SoA approaches, both quantitatively and qualitatively. I do have some concerns with this paper:
- The method is not explained clearly. For example, section 3.4 on progressive training only states a pre-trained Event3DGS is used. What does this mean? This is similar in other places in the paper.
- No method ablations are presented. It would be great to see the impact of the contributions, and this would show if a more  simple version would achieve the same result.

There are also some papers not mentioned which I believe should be part of the camera ready:
"EvaGaussians: Event Stream Assisted Gaussian Splatting from Blurry Images" - https://arxiv.org/abs/2405.20224
"Ev-GS: Event-based Gaussian splatting for Efficient and Accurate Radiance Field Rendering" - https://arxiv.org/abs/2407.11343 - this work came out after the submission deadline and does not have to be considered a baseline.
questions: questions: Please consider my comments above.
limitations: limitations: 1
review_summary: review_summary: To the best of my knowledge, this is currently the SoA for 3D reconstruction using event-based cameras. While the writing might be lacking and the method is not entirely novel, I believe it should be accepted. It has clear applications in robotics.
title: title: Replacing EventNeRF’s NeRF with Gaussian Splatting
paper_summary: paper_summary: This paper tackles the task of 3D reconstruction using event-based cameras. Following EventNeRF, which used a loss between the rendered and integrated differential signals, this work trains a 3D scene representation. The key contribution is to replace the NeRF backend with a 3D Gaussian Splatting one while figuring out key implementation details e.g. progressive training instead of relying on SfM initialization. The resulting approach is significantly faster than EventNeRF while also being more accurate.
main_review: main_review: Positives:
+ The empirical results are certainly promising. The proposed approach is both, faster and more accurate than the relevant baseline.
+ Results are presented across both, real-world and synthetic settings.
+ The presented approach allows for a minor modification to handle motion blur, and this is show to be effective (although this is orthogonal to the gaussian splatting)
+ The ablations do show the benefit of progressive initialization

Concerns:
- The technical contributions are a bit limited. In essence, the paper seeks to replace the 3D representation in a prior work to a currently more popular one, and there really aren't any interesting technical innovations in doing so.

- The connection to robotics is rather tenuous. The experiments are all in synthetic or self-collected setups, and not really in a scenario that a reader can directly connect to robotics. Moreover, the approach assumes known camera poses, which itself is a huge challenge in high-speed ego-motion settings.

As a minor point, the paper title should replace ‘for’ with ‘under’ because the paper does not really show any use of the reconstruction for the high-speed motion.
questions: questions: - The GS paper had two losses in RGB space which this work adopts, but are both these still needed even with EventGS?
- How are the test viewpoints selected? Are these a subset of the training viewpoints (albeit with the caveat that their RGB images were not directly used)?
limitations: limitations: 2
review_summary: review_summary: Overall, while the empirical results for reconstruction/view synthesis are promising, the lack of technical contributions and questionable relevance to robotics prevent a recommendation towards accept. Edit: Leaning towards accept after author response.
title: title: Good result but marginal contribution on top of the formulation of EventNerf + 3DGS backbone and no explanation on why it works better.
paper_summary: paper_summary: The paper proposes an event-based 3DGS that leverages the high-temporal resolution of the event camera.
main_review: main_review: ### Strengths

- Good performance in various datasets.
- Significantly faster in training/inference thanks to 3DGS.
- Easy to read and straight forward method.

### Weaknesses

- Not much technical contribution and explanation.
  - The formulation is very similar to the baseline method (EventNerf [9]). Why is the propose method works better? Could you provide equivalent version of Nerf or 3DGS trained with the EventNerf loss?
  - If the different loss is the contributing factor, the ablation study should be included.
  - Comparison to different slicing approaches to verify the proposed approach is contributing to the performance (e.g. EventNerf uses randomized length slices).
- Important sections do not have detailed explanation.
    - How do you get the camera poses and are they updated during training? Are they accurate from blurry images? If the poses are relatively accurate, can we use the sparse pointcloud from the pose estimation as the initial points?
    - Progressive training: how is this different from the conventional 3DGS training?
    - How are the parameters are chosen: $\sigma_{noevt}$, gamma correction parameter $g$, and $\Delta$ and how sensitive they are?
- Needs more baselines or experiments.
questions: questions: Most of the questions are addressed in the review section. One additional question: how is $g$ global parameter? Are all the data use the same event camera? If $g$ is a coefficient to convert from integrated events to an image, my understanding is that this should be varying by camera.
limitations: limitations: 2
review_summary: review_summary: Even with lack of explanations, it combined EventNerf and 3DGS with proper adaptations and proposed a faster and better model.