Title: Spike4DGS: Towards High-Speed Dynamic Scene Rendering with 4D Gaussian Splatting via a Spike Camera Array
Venue: NeurIPS 2025
Accepted: yes
Keywords: High-speed scene; Novel view synthesis; Multiple views; Spike camera
Abstract: Spike camera with high temporal resolution offers a new perspective on high-speed dynamic scene rendering. Most existing rendering methods rely on Neural Radiance Fields (NeRF) or 3D Gaussian Splatting (3DGS) for static scenes using a monocular spike camera. However, these methods struggle with dynamic motion, while a single camera suffers from limited spatial coverage, making it challenging to reconstruct fine details in high-speed scenes. To address these problems, we propose Spike4DGS, the first high-speed dynamic scene rendering framework with 4D Gaussian Splatting using spike camera arrays. Technically, we first build a multi-view spike camera array to validate our solution, then establish both synthetic and real-world multi-view spike-based reconstruction datasets. Then, we design a multi-view spike-based dense initialization module that obtains dense point clouds and camera poses from continuous spike streams. Finally, we propose a spike-pixel synergy constraint supervision to optimize Spike4DGS, incorporating both rendered image quality loss and dynamic spatiotemporal spike loss. The results show that our Spike4DGS outperforms state-of-the-art methods in terms of novel view rendering quality on both synthetic and real-world datasets. More details are available at https://github.com/Qinghongye/Spike4DGS.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes using a multi-view spike camera array to synthesize novel views of dynamic scenes. The authors first construct both synthetic and real-world multi-view spike datasets and introduce a GS initialization module that directly estimates dense point clouds and camera poses from spike streams, without relying on existing SfM tools like COLMAP or DUST3R. The framework further integrates 4D Gaussian Splatting for high-speed dynamic scene modeling.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\nA new dataset with multi-view spike camera data covering both synthetic and real-world scenes is introduced, which is valuable for further research.\n\nCompared with traditional two-stage methods that first reconstruct images before applying 4DGS, the proposed approach shows better performance, demonstrating the benefit of joint modeling.\n\nWeaknesses\n\nAlthough spike cameras offer high temporal resolution, this advantage is neither fully exploited in the framework design nor clearly reflected in the experimental results. \n\nOn the synthetic dataset shown in Figure 3, all methods achieve suboptimal results. It raises the question of whether image-based NVS methods (e.g., COLMAP + 4DGS) suffer from similar artifacts on this data. \n\nThe authors claim that traditional methods rely on high-quality image reconstruction (Line 140), but the proposed method also requires reconstructed images (Line 170), and the deformed Gaussians in Section 3.4 are highly dependent on image quality. The spike signals only serve as auxiliary loss. \n\nWhile several spike-based NVS methods are mentioned in Section 2.2, they are not directly compared in experiments. Can those methods be adapted to the multi-view setting? This deserves further discussion. \n\nThe proposed multi-view spike-based dense initialization lacks clarity in what makes it unique to spike data. If the input were replaced with multi-view images, the module might still function. \n\nIt remains unclear what the core advantage over the \"spike-to-image + image 4D NVS\" pipeline is: better initial point clouds and poses? Higher-quality reconstructed images? Or improved supervision through spike constraints?\n\nQUESTIONS:\nHow does the spike-based dense initialization compare to initialization based on original images (e.g., COLMAP)? Table 3 only compares against spike-reconstructed images, which are of low quality.\n\nWhile some inaccuracy in point cloud is acceptable for initialization, it would be informative to provide some evaluations of point clouds and camera poses, especially compared to COLMAP and DUSt3R.\n\nWhen using MSDI for initialization, are the poses further refined in 4DGS stage? Is the pose accuracy comparable to COLMAP-based pipelines?\n\nWhat is the computational cost of the proposed method?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper constructs an array of spike cameras and uses it to capture multiview images of a dynamic scene. With this novel dataset in hand, the authors then develop and apply 4D Gaussian splatting reconstruction framework to reconstruct a dynamic volumetric reconstruction of the scene. The proposed reconstruction algorithm has a novel initialization scheme and loss function. The initialization scheme estimates camera poses and an initial point cloud from an input spike stream using a vision transfomer. The loss function consists of (1) a spike loss between the capture data and spikes rendered by the reconstruction and (2) a pixel loss between images reconstructed directly from the spike cameras and those rendered by the reconstruction. The proposed method outperforms existing methods on real and simulated data.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths:\n\nTo my knowledge, the proposed method and camera array are novel\n\nThe dataset, assuming it is publicly released, would be useful\n\nThe proposed method significantly outperforms existing methods.\n\n## Weaknesses:\n\nMissing Evidence: The paper provides limited evidence that 3D reconstruction was successful. While tables 2-4 in the supplement report novel view reconstruction accuracy on simulated data, there are no \"fly-through\" in the supplement, depth maps, or other qualitative evidence showing that accurate 3D geometry was recovered. The demo videos are rendered from a fixed perspective and fail to demonstrate any 3D geometry was successfully recovered. It's unclear if the results shown in the paper are rendered from a novel view point or one that was already captured by one of the cameras.\n\nClarity and missing information: The main text shows the cameras positioned in such as way that their viewpoints wouldn't overlap and 3D reconstruction would be extremely challenging. The supplement suggests these cameras were positioned with overlapping views. Please provide information about the position, orientation, FoV, etc. of each camera. \n\nIn the abstract and title, the paper uses the word \"rendering\" when what is really meant is \"reconstruction\".\n\nQUESTIONS:\nCould the authors provide qualitative evidence that the method is successfully reconstructing 3D geometry? E.g., a visualization of the 3DGS depth map or views rendered from a sequence of perspectives that were not originally captured. This is my major criticism of the current manuscript---it claims 3D reconstruction but doesn't convincingly demonstrate it.\n\nAre Figures 3, 4, and 5 rendered from novel viewpoints or viewpoints that were already captured? How close are the rendered views' perspectives to an existing view's?\n\nWill the datasets be publicly released? The paper lists the dataset as a contribution, suggesting it will be, but this should be made more explicit.\n\nDid the cameras have overlapping view points? (In the main text they are positioned s.t. their views likely wouldn't overlap.) If the viewpoints are not overlapping and the camera is not moving, how is 3D reconstruction being performed?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes Spike4DGS, the first method for reconstructing high-speed dynamic scenes using a spike camera array. The approach begins with a Multi-view Spike-based dense initialization to generate an initial point cloud, estimate camera poses, and synthesize RGB images from the spike streams. These synthesized images and estimated poses are then used to optimize 4D Gaussian Splatting (4DGS). To further enhance reconstruction quality, the method incorporates a novel Spike Variation Supervision mechanism that leverages the unique characteristics of spike data. Additionally, the authors create two benchmark datasets, one synthetic and one real-world, to evaluate the method’s performance. Extensive experiments demonstrate that Spike4DGS achieves superior results compared to existing baselines.\n\nSTRENGTHS AND WEAKNESSES:\nStrength\n1. The paper introduces Multi-view Spike-based Dense Initialization (MSDI) to jointly initialize the point cloud and estimate camera poses directly from spike streams, while simultaneously recovering images. This unified approach replaces the traditional two-step pipeline: first recovering images and then performing point cloud initialization, thereby reducing reliance on spike-to-image conversion methods, which may produce suboptimal results.\n2. The authors introduce one synthetic and one real-world dataset as benchmarks for evaluating high-speed dynamic scene reconstruction from spike streams. These datasets may serve as valuable resources for the community and help advance future research in spike-based reconstruction.\n3. A spike variation loss is introduced to exploit the rich texture and motion cues inherent in the spike stream, thereby enhancing the quality of the reconstructed scenes.\n4. The paper is well-written and easy to follow.\n\n\nWeakness\n1. All baseline methods follow a two-stage pipeline of spike-to-image conversion followed by dynamic reconstruction. Would it be more appropriate to consider a stronger baseline by directly modifying existing spike-based methods such as SpikeGS or SpikeNeRF to support dynamic rendering—e.g., by integrating a deformation field? This could provide a more direct and fair comparison.\n2. A visual and quantitative comparison between the RGB images generated by spike-to-image methods (e.g., TFI, TFP, spk2img) and those produced by the proposed MSDI module would better validate the effectiveness of the spike-based initialization.\n3. In Figure 5, the rendering quality degrades noticeably as the number of input views decreases. It would be helpful for the authors to clarify how many input views were used in the comparisons with baseline methods. Additionally, providing a comparison with baselines under varying numbers of input views would offer valuable insight into the robustness of the proposed method in sparse-view settings.\n\nQUESTIONS:\nSee weakness 1, 2, 3.\n\nLIMITATIONS:\nDiscussed in supplmentary section I.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a novel framework for high-speed dynamic scene rendering. It uses an array of spike cameras to capture dynamic scenes at very high temporal resolution to reduce motion blur, and then applies 4D Gaussian Splatting (4DGS) for Novel View Synthesis (NVS). The core components include a Multi-view Spike-based Dense Initialization (MSDI) module, which converts spike camera streams into 4DGS readable formats, and a Spike-Pixel Synergy Supervision (SPSS) strategy, which enables direct 4DGS supervision using spike signals.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. This paper thoroughly explains the current limitations in dynamic scene rendering, particularly the motion blur introduced by conventional low-speed cameras.\n2. The idea of using a spike camera array to alleviate motion blur and enable multi-view NVS is intuitive, reasonable, and novel.\n3. The proposed MSDI and SPSS modules, if fully evaluated, have the potential to support many 3DGS-related downstream tasks.\n4. Quantitatively, although there is no prior work with an identical setup, the reported metrics surpass the SOTA in dynamic NVS tasks by a significant margin.\n\nWeaknesses:\n1. The Methodology section lacks clarity. The authors simply list a series of equations without explaining the intuition or reasoning behind these design choices, making the section very difficult to follow.\n2. There is little to no analysis in the Experiments section. It is unclear to the reviewer which components actually contribute to the large performance gains, raising questions about the setup for other methods.\n3. Limited generalizability: The real-data experiments only cover small-scale, toy examples. The authors also mention providing ideal lighting conditions when collecting real data. Taken together, these factors raise questions about the pipeline’s ability to handle datasets outside their own collection.\n4. Too many typos: The paper contains way too many typos and grammatically incorrect sentences, making it quite distracting to read.\n\nQUESTIONS:\nThe selling points of the paper seem to be the MSDI and SPSS modules, but I am having a hard time assessing their claimed effectiveness. So, my questions are around the method design and evaluation:\n1.\tEquation (17): Can you explain why you chose the total variation loss rather than an L1 or L2? A brief justification of this design choice would help clarify its role.\n2.\tMSDI is very interesting to me. It would be helpful if the authors could provide a more detailed evaluation of this module—for example, by feeding the MSDI outputs (point clouds, reconstructed images, and poses) directly into 4DGS without SPSS. This would help identify whether the gains come primarily from better pointclouds/poses (versus DUSt3R) or better images (versus spk2img).\n3.\tCan you also try replacing only the initialization pointcloud with one from DUSt3R or COLMAP while keeping everything else the same? This would help isolate the contribution of your initialization strategy as any Gaussian Splatting based rendering method is greatly reliant on the initial pointcloud input.\n4.\tCould you showcase some examples in more realistic settings—such as a car driving down a road in real lighting conditions? I understand that your pipeline assumes uniform lighting, but seeing examples in real-world scenarios would help reviewers better assess its effectiveness and generalizability. I may reconsider my final rating depending on these results.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes Spike4DGS, the first high-speed dynamic scene rendering framework with 4D Gaussian Splatting using spike camera arrays. 3DGS from conventional cameras is fundamentally limited by framerate, and therefore struggle with fast-moving scenes. Spike cameras asynchronously encode absolute light intensity into continuous spike streams at rates of up to 20k Hz. However, prior works with spike cameras are limited to 3DGS: thereby suffering in dynamic scenes. \n\nTo solve this, the input multi-view spike streams are sent to a Multi-view Spike-based Dense Initialization to estimate point cloud and camera poses. MSDI consists of ViT based encoders which are fused to generate a point cloud initialization and pose estimates. The features are also used to generate discrete images from the spike stream. Using these as inputs to a 4DGS generation pipeline, a 4DGS model may be generated. To better supervise 4DGS training, in addition to the pixel level loss, rendered images are also converted to spike streams, which can be supervised against the original spike stream.\n\nThe authors propose both a synthetic and real multi-view dataset of spike streams. On this dataset, the proposed method outperforms comparison methods in terms of novel view rendering quality on both synthetic and real-world datasets. Ablation studies show the benefit of various components in the method.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The paper is written well, and it is reasonably straightforward to understand the method and contributions\n2. The motivation for the method, in terms of capturing high-dynamic scenes using spike cameras, is well described\n3. This paper proposes the first 4DGS method for spike cameras\n4. Results show superior performance against compared baselines\n5. Ablations motivate the need for various method components well\n\nWeaknesses:\n1. Since this is a 4DGS method, I think it is imperative to see several novel views for a given scene to show that the generated 3D representation is indeed robust, which I could not find in the paper.\n2. Overall, the method seems largely sequential. First, spike camera views are converted into traditional RGB camera representations (images, point clouds, camera poses etc.), and these are passed through an existing 4DGS method. So it seems the major novelty is the conversion of spike camera views to RGB views.\n3. The training camera views are all planar and therefore cannot truly represent the entire 3D scene, meaning the 3D representation quality on faraway camera views cannot be tested. \n4. It is not clear what type of scenes this method is good at representing vs what types of scenes it is not good for: the limitations section should incorporate this.\n\nQUESTIONS:\n1. The text in the figure 4 results seems flipped. Why is this?\n2. Can we have some sense of training time, and how it compares with baseline methods?\n3. Is there any reason why all the cameras were arranged in a planar manner?\n\nLIMITATIONS:\nYes, with some suggestions in the answers above", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-08"}