Title: HetSyn: Versatile Timescale Integration in Spiking Neural Networks via Heterogeneous Synapses
Venue: NeurIPS 2025
Accepted: yes
Keywords: Spiking Neural Networks

Synaptic Heterogeneity

Multi-Timescale Integration

Neuron Modeling
Abstract: Spiking Neural Networks (SNNs) offer a biologically plausible and energy-efficient framework for temporal information processing. However, existing studies overlook a fundamental property widely observed in biological neurons—synaptic heterogeneity, which plays a crucial role in temporal processing and cognitive capabilities. To bridge this gap, we introduce HetSyn, a generalized framework that models synaptic heterogeneity with synapse-specific time constants. This design shifts temporal integration from the membrane potential to the synaptic current, enabling versatile timescale integration and allowing the model to capture diverse synaptic dynamics. We implement HetSyn as HetSynLIF, an extended form of the leaky integrate-and-fire (LIF) model equipped with synapse-specific decay dynamics. By adjusting the parameter configuration, HetSynLIF can be specialized into vanilla LIF neurons, neurons with threshold adaptation, and neuron-level heterogeneous models. We demonstrate that HetSynLIF not only improves the performance of SNNs across a variety of tasks—including pattern generation, delayed match-to-sample, speech recognition, and visual recognition—but also exhibits strong robustness to noise, enhanced working memory performance, efficiency under limited neuron resources, and generalization across timescales. In addition, analysis of the learned synaptic time constants reveals trends consistent with empirical observations in biological synapses. These findings underscore the significance of synaptic heterogeneity in enabling efficient neural computation, offering new insights into brain-inspired temporal modeling. Code available at: https://github.com/dzcgood/HetSyn.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces HetSyn, which incorporates synaptic heterogeneity into spiking neural networks (SNNs) to enable multi-timescale integration. The authors instantiated HetSyn in combination with LIF neurons as HetSynLIF and experimentally validated it in pattern generation, delayed match-to-sample, speech recognition and visual recognition tasks.\n\nSTRENGTHS AND WEAKNESSES:\nStrength:\n1. HetSyn introduces synaptic heterogeneity into the SNN, which is consistent with biological mechanisms.\n2. The authors analyze the proposed HetSynLIF and point out that other LIF neuronal variants that exist can be treated as special cases of HetSynLIF.\n3. The authors demonstrated the effectiveness of the proposed method across multiple tasks.\n\n\nWeakness:\n1. The details of HetSynLIF need to be described in more detail, such as how the synaptic time constants are set and their impact on performance; if the synaptic time constants are learnable, what is the evolutionary trend during training?\n2. Could the theoretical contribution of HetSynLIF to the spatio-temporal representation performance of SNNs be analyzed in depth?\n3. The authors should analyze the impact of setting up individual time constants for each synapse on the overhead of the SNN, as this incurs a large number of additional parameters.\n4. The experiments in this paper are limited to tiny taskss, making it difficult to demonstrate the effectiveness of the proposed method for challenging tasks and its scalability. Does HetSynLIF work for typical visual recognition tasks, such as CIFAR and ImageNet? Furthermore, [1] achieved significantly better performance on SHD, even with a single-layer SNN. How does this justify the performance advantage of HetSynLIF?\n\n[1] Advancing Spatio-Temporal Processing in Spiking Neural Networks through Adaptation. arXiv. 2024.\n\nQUESTIONS:\nSee weaknesses.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-27"}

comments: SUMMARY:
The authors propose the use of spiking neural networks (SNNs) for classification or temporal signal generation tasks, enhanced by introducing heterogeneity in synaptic time constants. They demonstrate that these networks outperform conventional networks lacking this variability.

STRENGTHS AND WEAKNESSES:
One of the notable strengths of this paper is its foundation in physiological data, illustrating how such biological inspiration can enhance the performance of spiking neural networks. The performance improvements are evident across the tasks presented, with classification results that surpass those of similar networks (RSNNs) without heterogeneity. The application to a delayed match-to-sample task is particularly compelling.

A limitation of this study is its incremental nature compared to previous work that has explored variability in SNNs, such as citation 22. Additionally, the paper lacks an interpretation of the parameters obtained after training, which could provide insights into the underlying mechanisms driving the improved performance.

QUESTIONS:
When training your network on the delayed match-to-sample task, can you interpret the parameters obtained post-training? Specifically, in the distribution of time constants, do you observe values on the order of the delay duration (around 800 milliseconds)?

The state-of-the-art performance for the SHD dataset has been achieved using spiking neural networks with delays. Do you believe it would be feasible to combine these two approaches to further enhance performance?

LIMITATIONS:
A limitation of this work is its relatively incremental progress compared to existing research, particularly citation 22.
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 2
DATE: 2025-06-27

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work proposes HetSyn, a variant of a spiking neural network where the synaptic time constant of each synapse is treated as an independent trainable parameter. The authors applied the proposed model to four different tasks and demonstrated that it outperforms the model where only the neuron-level membrane time constant is trainable and models without plastic decay time constants.\n\nSTRENGTHS AND WEAKNESSES:\nStrength: \n\nThe numerical experiments clearly demonstrate the advantage of the proposed model against the model with adaptive membrane time constant in pattern generation, working memory, and speech recognition tasks.  \n\nGiven the popularity of the membrane time constant adaptation in spiking neural network literature (e.g., Fang et al., ICCV 2021), I thought trainable synaptic time constant has also been implemented before, but I couldn’t find a previous work that specifically implements this variant. Several works previously introduced trainable synaptic delays (see the reference below), but these mechanisms are mathematically distinct from trainable synaptic time constant introduced here.\n\nWeakness: \n\nThe data analysis presented in Fig. 1A is misleading from a neuroscience perspective. In excitatory synapses in the cortex, there are two neuro-transmitter types: AMPA and NMDA, having different decay timescales. AMPA decays fast in a few milliseconds, while NMDA decays slowly in the order of 100 milliseconds. Fig. 1A shows a bimodal distribution which presumably corresponds to AMPA and NMDA components. The claim of ‘mildly long-tailed distribution’ is somewhat odd, given this underlying biology. Additionally, the importance of having these two neurotransmitters with different timescales has been studied extensively in computational neuroscience literature (e.g., Brunel and Wang, J Comp Neurosci, 2001; Panzeri et al., Trends Neurosci 2010).     \n\nSome of the comparisons are not well controlled. For instance, in Fig. 2D, the authors claim that the HetSynLIF model converges faster than the other models. However, the comparison was performed at a single learning rate; thus it does not rule out the possibility that this particular learning rate is optimized for HetSynLIF over others. To claim faster convergence, the hyperparameters of each model should be optimized individually for their best performance for a fair comparison.\n\nQUESTIONS:\nHave you compared the proposed model with the previous models where synaptic delays are trained? In what kind of tasks do you think the proposed model performs better?\n\nRefs: \n\nZhang, Malu, et al. \"Supervised learning in spiking neural networks with synaptic delay-weight plasticity.\" Neurocomputing 409 (2020): 103-118.\n\nYu, Qiang, et al. \"Improving multispike learning with plastic synaptic delays.\" IEEE Transactions on Neural Networks and Learning Systems 34.12 (2022): 10254-10265.\n\nLIMITATIONS:\nPlease see the comments above", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe author proposes a framework HetSyn for modeling prominent heterogeneity by combining the widely observed but underutilized characteristic of synaptic heterogeneity in biological neurons. This framework combines synaptic specific time constants to reflect synaptic heterogeneity and is instantiated through HetSyn LlF, achieving time integration across different time scales at the synaptic level. This model takes several existing types of neurons (such as Alif, heterogeneous LIF) as special cases and evaluates them on multiple tasks, including pattern generation, delayed matching samples, samples, speech recognition (SHD), and visual recognition.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. Introducing synaptic heterogeneity to achieve versatile timescale integration of SNNs, based on empirical neurobiological evidence. This contributes to computational neuroscience and neuromorphic computing.\n2. Comprehensive evaluation, spanning multiple tasks and datasets. The design of ablation research and robustness testing (noise, time distortion, resource constraints) is reasonable.\n3. The paper provides theoretical analysis for the proposed method.          \n\nWeaknesses:\n1. Introducing trainable specific synaptic time constants can significantly increase the number of parameters and may increase the complexity of optimization. Suggest quantifying training costs (such as power consumption, memory, time) and comparing them with existing work.\n2. This paper lacks an evaluation of event based datasets in the real world, such as DVS Gesture or DVS-CIFAR10. Given that the model emphasizes the temporal dynamics of synapses, testing it on asynchronous, event driven data will better demonstrate its practicality and robustness.\n\nQUESTIONS:\n1. Can author quantify training costs (such as power consumption, memory, time) and comparing them with existing work.\n2. Can the authors either provide preliminary results on event-driven benchmarks or clearly justify why these were excluded? Would the model need adaptation to handle asynchronous input?\n\nLIMITATIONS:\nThe Limitation section is brief. It is recommended to discuss potential limitations, such as introducing time constants for each synapse, which significantly increases the optimization dimension and may lead to gradient instability or slow convergence.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}