Title: High Dynamic Range Imaging with Time-Encoding Spike Camera
Venue: NeurIPS 2025
Accepted: yes
Keywords: Spike Camera

High Dynamic Range

Imaging
Abstract: As a bio-inspired vision sensor, spike camera records light intensity by accumulating photons and firing a spike once a preset threshold is reached. For high-light regions, the accumulated photons may reach the threshold multiple times within a readout interval, while only one spike can be stored and read out, resulting in incorrect intensity representation and a limited dynamic range. Multi-level (ML) spike camera enhances the dynamic range by introducing a spike-firing counter (SFC) to count spikes within each readout interval for each pixel, and uses different spike symbols to represent the arrival of different amounts of photons. However, when the light intensity becomes even higher, each pixel requires an SFC with a higher bit depth, causing great cost to the manufacturing process. To address these issues, we propose time-encoding (TE) spike camera, which transforms the counting of spikes to recording of the time at which a specific number of spikes (i.e., an overflow) is reached. To encode time information with as few bits as possible, instead of directly utilising a timer, we leverage a periodic timing signal with a higher frequency than the readout signal. Then the recording of overflow moment can be transformed into recording the number of accumulated timing signal cycles until the overflow occurs. Additionally, we propose an image reconstruction scheme for TE spike camera, which leverages the multi-scale gradient features of spike data. This scheme includes a similarity-based pyramid alignment module to align spike streams across the temporal domain and a light intensity-based refinement module, which utilises the guidance of light intensity to fuse spatial features of the spike data. Experimental results demonstrate that TE spike camera effectively improves the dynamic range of spike camera.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nTo solve dynamic range limitations of conventional spike cameras, and the high energy consumption of ML spike cameras, the authors propose a time-encoding based spike camera. This camera transforms counts into a time when an overflow is reached. The authors then propose a reconstruction method for recovering traditional imagery from the TE spikes. The method is compared against ML spike cameras and various alternative reconstruction algorithms.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n-\tThe paper thoroughly describes how a novel time-encoded spike cameras might operate.\n-\tThe comparison against a ML spike camera is complete and fair by examining on two datasets with many different software reconstruction algorithms. \n-\tThe ablation study shows why each part of the proposed reconstruction algorithm is useful.\n\nWeaknesses\n\n-\tComparison is limited to spike camera technology. Also, advantages or motivation of using a TE spike camera instead of any of the other methods described in the related work is missing.\n-\tEvaluation is limited to simulation under idealized settings where the only noise is due to intrinsic shot noise and not any other hardware inconsistencies.\n-\tMany figure & table captions are short making them more difficult to understand and what the reader should take away. For instance, figure 3 is confusing because an arrow from the first box flows into (c) suggesting this is one large flow diagram, but that is not the case.\n\nQUESTIONS:\n1. Over what time interval are spikes passed to the reconstruction network? And how fast does the network recover full frame HDR images?\n2. What is the motivation or advantage of using a spike camera for HDR imagery over the alternatives? For instance, SPADs and event cameras are equally as fast. Why is recovering a full frame necessary for applications such as autonomous driving where the application is something else like obstacle avoidance?\n3. What is the final dynamic range of the proposed system?\n4. What is the average bandwidth / energy requirement of the proposed system versus the ML spike camera system?\n\nLIMITATIONS:\nNo method limitations are discussed.\n- Some spike camera crops look blurry in comparison to the GT image.\n- Only grayscale imagery is recovered, but for consumer applications where recovering a full frame is necessary, color is often important.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces time-encoding (TE) spike cameras, which address limitations of existing multi-level (ML) spike cameras. Rather than storing the number of spikes that are fired during each readout interval, TE spike cameras record the time required until a set number of spikes have fired. This is beneficial for scenes with high intensity content, where high bit-depth and power are required to count the large number of spikes, because TE spike cameras can encode HDR content with less bit-depth than their ML counterpart. The authors use a neural network to extract features, perform frame alignment, and refine the reconstruction result.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n1. The paper proposes a novel temporal encoding method that offers clear benefits over conventional ML spiking approaches.\n2. Key claims and network architecture choices are empirically supported. Simulated experiments provide evidence that TE data encodes more information than ML data. The ablation study provides information about the importance of each network component.\n\nWeaknesses\n1. The experimental results are performed in simulation only.\n\nQUESTIONS:\n1. I would like to see more concrete examples given in your discussion of \"fast ultra-HDR\" scenes to highlight the importance and impact of TE spiking cameras. From the current discussion, it's not immediately apparent which applications this method would be most beneficial for.\n2. Are there any challenges from a practicality standpoint in building the proposed TE spiking camera? Are there any tradeoffs (or increase in complexity) for building a TE vs. ML spiking camera?\n3. Representing spike streams as a large volume $\\mathbb{R}^{H \\times W \\times T}$ is inefficient since (presumably) much of the data in the stream will be zeroes (i.e., no spikes triggered). How does this affect network training / inference at scale? Can you run directly on spike data to save memory / computation?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nSpike cameras offer high readout frequency but imaging HDR scenes using spike cameras remains an open problem. Prior work uses multi-level (ML) spike cameras to denote multiple spikes fired within the time frame of a single readout to increase dynamic range, while the authors claim that counting a large number of spikes can be costly in ultra-HDR scenes. To address this, the authors propose a time-encoding (TE) which records the time it takes for the spike count in every pixel to reach the maximum, thereby further increasing the dynamic range. They also propose a reconstruction method for the proposed TE spike data. Experimental results revealed that reconstructed TE spike data provides superior quality to reconstructed ML spike data.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\nThe problem of interest, imaging HDR scenes with spike cameras, is a relatively new topic and worths investigating. The proposed TE spike camera is well motivated and the design is reasonable. According to the experiments, when the same reconstruction method is applied, the proposed TE spike camera consistently offers higher image quality than the existing ML spike camera. The proposed reconstruction method outperforms the other reconstruction methods investigated.\n\nWeaknesses:\nIt is not entirely clear to me how to understand TE spike cameras with reference to ML spike cameras. Do TE cameras provide higher dynamic ranges at the cost of additional resources, or do they present a better use of the circuitry and balance of bit depths? To image the same scene with a fixed dynamic range using the two types of cameras, how can we compare their energy consumption, bandwidth, latency, reconstruction difficulty etc.? In addition, the operating regimes of the two types of spike cameras in terms of dynamic range or bit depth need to be clarified.\n\nThe reconstruction section is somewhat disconnected from the proposal of TE spike cameras. It would be better if the authors could offer more insights into the rationale behind the method design.\n\nAccording to Fig 4, the visible performance gain is mainly on motion mitigation instead of dynamic range recovery. This does not align with the focus of this paper as introduced in the previous sections. Furthermore, whether the test data aligns with the ultra-HDR theme is not explained. While the TE results present higher scores and fewer blur effects, the advantages of TE spike cameras over ML ones are not fully verified by the results.\n\nThis core contribution of this work is the proposal of a new design of spike cameras, but no prototype camera or real-world experiment is presented to verify the effectiveness and practicality of the proposed device.\n\nQUESTIONS:\nCan the authors comment on the operating regimes of the types of spike cameras, i.e., what dynamic range is considered HDR where ML spike cameras are applicable and what is considered ultra-HDR where TE spike cameras are needed?\n\nWhat is the bit depth and the dynamic range of images in the dataset used? How does it align with the HDR and ultra-HDR arguments?\n\nIn the experiment, ML spike cameras use an 8-bit SFC while TE spike cameras use a 5-bit SFC plus a 4-bit CCC. Is this setting fair?\n\nLIMITATIONS:\nThe limitation is only briefly mentioned in a single sentence.", "TITLE": "Official Review", "RECOMMENDATION": 2, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nSpike cameras count incoming photons and emit a binary “spike’’ whenever a preset threshold is reached.  In very bright regions multiple spikes can occur inside one read-out period, but only a single bit is stored, so the dynamic range collapses.  The authors propose a time-encoding (TE) spike camera in which the sensor stops counting after every *b* spikes (an “overflow’’) and records when that overflow occurred by sampling a high-frequency periodic timing signal.  Storing the cycle index instead of every spike both increases dynamic range and reduces energy caused by rapid firing .\n\nFor reconstruction, they introduce a learning-based pipeline that (i) extracts multi-scale gradient features from the spike stream, (ii) aligns temporally adjacent streams with a similarity-based pyramid alignment (SPA) module, and (iii) fuses spatial features via a light-intensity refinement (LIR) module.\n\nOn two synthetic HDR benchmarks (HDM-HDR-2014 and Kalantari13) the TE camera plus their network improves PSNR by 3dB and SSIM by ≈0.05 over the same architecture fed with multi-level (ML) spike data, and beats the prior MambaSpike baseline by 2–6 dB .  Ablations show each module contributes to the gain.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n1. Recording the moment of overflow is a clever way to encode high luminance while keeping digital bandwidth constant.\n2. Use of gradient-aware features and pyramid alignment match the temporal sparsity of spike data.\n3. Shows consistent improvement over both hardware (ML) and algorithmic baselines on two datasets .\n\nWeaknesses and Questions\n\n1. Lack of real-sensor evidence – all experiments are synthetic. No prototype TE chip or mixed-signal simulation is shown, so claimed energy savings remain hypothetical.\n2. Limited baseline set – comparison omits recent hybrid-sensor HDR methods that fuse events with frames or ML-based spike counters; fairness of gains is unclear.\n3. Reproducibility gaps – authors will release code only “after publication’’ and provide no error bars. Statistical significance is therefore unknown .\n4. Hardware feasibility – storing a high-frequency CCC per pixel may raise SRAM and routing pressure; silicon overhead is not analysed.\n5. Generalisation to real HDR noise – Poisson shot noise and hot pixels are not modelled in the synthetic data. Hence robustness might degrade in practice.\n\nQUESTIONS:\nSuggestions\n\n1. Provide at least SPICE or Verilog-A level simulations of pixel, CCC, and timing signal to quantify area/energy overhead versus ML spike counter.\n2. Include newer event/frame fusion HDR networks as baselines; retrain them on the same datasets to ensure fairness.\n3. Release training scripts and checkpoints at submission time and add confidence intervals (3 seeds) for key metrics.\n4. Analyse sensitivity to clock frequency choice f and overflow depth b; a plot of dynamic-range gain vs. energy would guide sensor designers.\n5. Test reconstruction on real spike-camera HDR sequences (e.g., hybrid SpikeCam + LED-flash dataset) to demonstrate domain robustness.\n\nLIMITATIONS:\nsee above in weaknesses", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}