Title: Dendritic Resonate-and-Fire Neuron for Effective and Efficient Long Sequence Modeling
Venue: NeurIPS 2025
Accepted: yes
Keywords: Resonate and Fire Neuron

Long Sequence Modeling

Spiking Neural Networks
Abstract: The explosive growth in sequence length has intensified the demand for effective and efficient long sequence modeling. Benefiting from intrinsic oscillatory membrane dynamics, Resonate-and-Fire (RF) neurons can efficiently extract frequency components from input signals and encode them into spatiotemporal spike trains, making them well-suited for long sequence modeling. However, RF neurons exhibit limited effective memory capacity and a trade-off between energy efficiency and training speed on complex temporal tasks. Inspired by the dendritic structure of biological neurons, we propose a Dendritic Resonate-and-Fire (D-RF) model, which explicitly incorporates a multi-dendritic and soma architecture. Each dendritic branch encodes specific frequency bands by utilizing the intrinsic oscillatory dynamics of RF neurons, thereby collectively achieving comprehensive frequency representation. Furthermore, we introduce an adaptive threshold mechanism into the soma structure. This mechanism adjusts the firing threshold according to historical spiking activity, thereby reducing redundant spikes while maintaining training efficiency in long-sequence tasks. Extensive experiments demonstrate that our method maintains competitive accuracy while substantially ensuring sparse spikes without compromising computational efficiency during training. These results underscore its potential as an effective and efficient solution for long sequence modeling on edge platforms.


Peer reviews:
comments: SUMMARY:
The authors describe the resonate and fire (RF) neuron in the context of being able to model longer term dependencies than the more conventional LIF neuron.  They propose a dendritic RF (D-RF) neuron where some of the dynamics are modelled by dendrites, allowing a single neuron to handle multiple resonances (one per dendrite).  They further propose an adaptive threshold that can be thought of as analogous to the spike frequency adaptation that is common in LIF implementations.
Experiments on basic but reasonable datasets yield good results.

STRENGTHS AND WEAKNESSES:
Strengths:

Paper structure is good; subject is timely and well motivated.
The RF architecture is relatively not well studied, so the general approach is quite novel.
Results are good.

Weaknesses:
The modifications to RF are in the spirit of functions that are known to work well on LIF neurons, so the novelty could be dismissed as obvious.
The datasets used are not especially "long" when compared to the things that are used for LLMs nowadays.  Then again, the LLM state of the art is well advanced whereas this is much more speculative.
It is not obvious whether the proposed architecture remains biologically plausible.

QUESTIONS:
One assumes (perhaps wrongly) that the original RF neuron was based on observation of biology.  To what extent do the proposed modifications retain (or not) that biological plausibility?

How does the sequence length compare with current long NLP baselines?  Speech input (at ~100 Hz sample rate) or repeated sampling of static examples surely represent long inputs, but how long is long?

LIMITATIONS:
No, I don't see limitations addressed explicitly in the paper.  However, the work is speculative enough to not really merit such an analysis.
TITLE: Official Review
CLARITY: 4
IMPACT: 4
ORIGINALITY: 3
DATE: 2025-07-01

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a D-RF neuron to address efficiency and performance challenges in long sequence modeling. The model comprises multi-dendritic branches and a soma structure, where each dendritic branch leverages RF neuron oscillatory properties to capture frequency-specific signals, while the soma employs an adaptive threshold mechanism that dynamically adjusts thresholds based on historical spiking activity. Experimental results demonstrate that D-RF achieves competitive performance across multiple long sequence tasks while maintaining sparse firing and training efficiency.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The paper provides a systematic analysis of RF neuron limitations, identifying constrained frequency response bandwidth and energy-efficiency trade-offs as key bottlenecks.\n2. The multi-dendritic architecture effectively expands frequency response capabilities, while the adaptive threshold mechanism facilitates efficient optimization.\n3. The adaptive threshold design maintains O(L log L) computational complexity, enabling practical SNN training for extended sequence modeling.\n4. Extensive experiments thoroughly validate the proposed method's effectiveness and efficiency.\n\nWeaknesses:\n1. Insufficient analysis of computational overhead analysis comparing D-RF to conventional spiking neurons.\n2. D-RF introduces additional complex number operations, and the authors did not discuss whether this feature hinders the deployment of neuromorphic chips.\n3. The selection of dendritic branch number lacks systematic analysis of the performance-complexity trade-off.\n\nQUESTIONS:\n1. Does introducing a multi-dendritic structure into LIF neurons enhance their temporal modeling capabilities? The authors are required to conduct experiments to verify this.\n2. D-RF neurons seem to introduce complex number operations. Will this affect their deployment in neuromorphic chips?\n3. How to train the RF neuron-based neural networks? The authors should clarify this question.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose the Dendrite Resonate-and-Fire (DRF) neuron as a novel approach to address the limitations of conventional Resonate-and-Fire (RF) neurons. To overcome the limited bandwidth response of a single RF neuron, the DRF architecture integrates dendritic branches with varying eigen-frequencies to capture complex frequency compositions. Additionally, the model incorporates a dynamic adaptive threshold mechanism with parallelization to balance training efficiency and spike sparsity. Empirical evaluations demonstrate that the DRF neuron achieves consistently improved performance across multiple time-series datasets, while maintaining comparable model size, preserving spike sparsity, and enhancing training efficiency.\n\nSTRENGTHS AND WEAKNESSES:\nStrength \n- Comparison of the time series dataset extensive and shows SOTA performance in all cases.\n- Faster training runtime and training energy cost efficient. Supports long sequence tasks. \n- Higher spike sparsity compared to other large spiking models on the LRA benchmark.\n\nWeakness\n- PathX is missing from the LRA Benchmark\n- The eigen-frequencies of the dendrites are fixed to the freq. bins of the input FFTs; in other words, depends on the discrete time delta. Would be interesting if these frequencies could be learned as well.\n- Neither hyperparameters nor the architecture of the models trained are mentioned in detail on the appendix or in the supplementary material; especially for S/PS-MNIST and SHD.   \n- (maybe too small a point to mention) Inconsistent number of parameters for S-CIFAR10 on Tables 7 and 8.\n\nQUESTIONS:\n1. How does the summation of dendritic signals at the soma contribute to frequency selectivity, given that the signals are merged before thresholding? How are multiple selected frequencies still reflected in the output spikes?\n\n2. What are the frequency initialization for the 8 dendrites that are shown in Figure 4 (c)? How is it possible to obtain enough information about the input signal with only 4-8 eigen-frequencies?\n\n3. What is the memory complexity of the D-RF network?\n\nLIMITATIONS:\nThey have not addressed the memory usage required for parallel processing, which may be high since the whole sequence of the data is input simultaneously.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 1, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes D-RF neurons to address the limitations of RF neurons in long sequence modeling. The work provides comprehensive theoretical analysis, revealing RF neurons' limited frequency band perception and energy-performance trade-offs through frequency domain analysis. The method is well-designed with D-RF employing multi-dendritic architecture for full-spectrum coverage and adaptive threshold mechanisms that balance training efficiency with sparse spike generation.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:  \n1. Comprehensive theoretical foundation: The method analyzes RF neurons' limited frequency band response and the challenge of balancing energy efficiency with training efficiency.  \n2. Efficient Neural Dynamics: D-RF neurons achieve full-frequency coverage through multi-dendritic structures while ensuring parallel training and sparse spike generation.  \n3. Competitive Performance: The method performs well on datasets of varying lengths, demonstrating efficient temporal modeling capabilities.  \n4. Trade-offs between efficient and train efficiency: Ablation studies demonstrate D-RF's effective balance between energy efficiency and training efficiency.  \n\nWeakness:    \n1. Compared to conventional spiking neurons [1, 2], D-RF neurons exhibit complex neural dynamics that introduce additional computational overhead.  \n2. Although the paper validates the method on mainstream sequence tasks, it lacks performance evaluation on speech datasets, neuromorphic datasets, and other relevant domains.  \n\n[1] Spiking neuron models: Single neurons, populations, plasticity[M]. Cambridge university press.  \n[2] Accurate and efficient time-domain classification with adaptive spiking recurrent neural networks[J]. Nature Machine Intelligence.\n\nQUESTIONS:\n1.To validate D-RF neurons' temporal processing capabilities, evaluation on speech datasets and neuromorphic datasets would be necessary.  \n2. Given the complex neural dynamics of D-RF neurons, providing pseudocode or algorithmic flowcharts would be beneficial for understanding and reproducibility.  \n3. The authors are encouraged to provide additional analysis of model complexity and performance variations as dendritic branches increase, to further justify the rationality of dendritic branch selection.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-07-03"}