Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The authors propose Gated LIF, a heterogeneous spiking neuron, to fuse different bio-features in different neuronal behaviors, enlarging the representation space of spiking neurons. They use gating factors to determine the proportion of the fused bio-features, which are learnable during training. The authors show that the combination of different learnable membrane-related parameters and methods can make spiking neurons constantly changing, thus increasing the heterogeneity and adaptivity of spiking neurons. 
strengths_and_weaknesses: Strengths:
 GLIF controls the fusion of different bio-features using different gating units for those three neuronal behaviors. GLIF can simultaneously contain different neuronal dynamic features, increasing the neuronal dynamic diversity of LIF models. 

The GLIF model consistently improves the performance of all the datasets and the models. An interesting thing to note is that GLIF consistently achieves 100% accuracy with a 0% error margin on the DVS Gesture dataset, which is impressive.




Weakness:
I felt the paper is generally a bit hard to read and very dense. Also, the structure of the paper's organization makes it confusing for readers unfamiliar with spiking neural networks. As such, it would be greatly appreciated if the authors could elaborate on the idea and make the contributions of the work a bit more explicit.

The author uses some terminology without explaining what it means, as it might be new for people from different backgrounds. For example, the main premise of the paper is based on the hypothesis that GLIF fuses different "bio-features" in different neuronal behaviors, expanding the representation space of spiking neurons. However, it is difficult to understand what the authors mean by "bio-features" as it has not been defined anywhere in the manuscript.
The author discusses the variation of the neuronal dynamics using Fig. 2 and Section 3.3, which seemed a bit redundant and can be moved to the Appendix. 

questions: As GLIF neurons add more gated functionalities to the standard LIF neurons, it would be great to see the computational cost and latency change due to the added computational overload.

The results shown in Tables 4-6 are very confusing - 
1. The results do not have any confidence interval - does that indicate that this is the best performance observed?
2. The results shown in Table 1 and Table 4 does not match - table 4 says the average accuracies are much higher than the accuracies on the same architecture and dataset shown in Table 4
3. Why did the authors choose T=4 for Table 4,5,6 when Table 1 clearly shows that T=6 consistently gives better results.

The authors in the ablation studies show the "Effectiveness of different bio-features." and "Importance of learnable gating factors" as separate sections. It would be great if the authors could discuss the difference between these two ablation studies and why they are essential.

In the ablation studies, the parameter sharing scheme is a bit hard to follow, and it would be great if the authors could add a bit more context and/or some references for the same.
limitations: The authors did not discuss the limitations of their work or the potential negative social impact of their work in the paper. The authors can discuss how the use of GLIF can make the SNN more biologically plausible, and by understanding the nature these SNN works, we can make more energy-efficient neural network models.
review_title: null

final_score: rating: 3: Reject: For instance, a paper with technical flaws, weak evaluation, inadequate reproducibility and incompletely addressed ethical considerations.
review_content: summary: The authors proposed three attention schemes that work on three different aspects of the LIF model: membrane potential leakage, integration accumulation, and membrane potential rest. The authors aim to enhance the LIF model's representation space with the proposed attention schemes. The key to the proposed attention schemes is three learnable parameters. They balance the pre-defined LIF behaviors, i.e., linear decay vs. exponential decay, uniform-coding vs. flexible-coding, and hard reset vs. soft reset rest. 

To show the effectiveness of the proposed method, the authors mainly leveraged ResNet-based architectures on classification tasks. 
strengths_and_weaknesses: Strengths:
+1. Enhancing SNNs' capability in encoding different bio-inspired behavior is an exciting direction. 

Weaknesses:
-1. I did not see the value of adding many learnable parameters into LIF. Essentially, they are still fixed during inferences and "support only a single biological feature in different neuronal behaviors." In addition, the SRM model has already offered much more parameters, which can be learned directly. As such, the authors should provide a comparison with SRM-based models.

-2. I think the key to SNNs research should be on the dynamic side. Dynamically dealing with different conditions is more appreciated. However, the proposed method did not introduce any dynamic schemes. 

-3. The authors applied tdBN to all their models. Did other competing models have tdBN? In addition, what's the performance without tbBN. Basically, we do not know whether the tbBN gives more contribution to the performance or the proposed method.

-4. How does the spiking threshold impact the classification performance? In theory, all the proposed attention schemes can be replaced by applying different spiking threshold to each SNN neuron. Therefore, it is necessary to show the experimental results related to different spiking thresholds. 
questions: writing issues:
1. line 41: "units $G_\alpha$, $G_\beta$, and $G_\gamma$ for those three neuronal behaviors" It is unclear to me which one is which. 
2. lines 58-59. Poor writing

limitations: From my perspective, the proposed method is incremental, not exciting. More importantly, the learnable parameters did not really solve the "single biological feature" problem. The proposed method added three dimensions to the LIF model but is still a static SNN model. 

The validation is weak from my perspective. Comparing with SRM is necessary, and the validation without tbBN is necessary. The experimental results related to the spiking threshold should also be reported and analyzed.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper proposes the GLIF neuron model, which has rich neuronal dynamics with learnable gates. The GLIF neuron achieves SOTA accuracy on CIFAR10, CIFAR100 and ImageNet datasets.
strengths_and_weaknesses: The complex spiking neuron model has not been well studied in deep SNNs. Most deep SNNs use the simple IF or LIF neuron. As far as I know, this is one of the first papers that focuses on the complex spiking neuron.

The design of the GLIF neuron is reasonable. And it gets SOTA accuracy on most datasets.

I notice that the proposed method gets 100% accuracy on DVS Gesture, which is hard to achieve. I check the codes and find that there is a bug at line 241 in data_builder.py: 

test_dataset = DVS128Gesture(dataset_dir, train=True

It seems that the author takes train set as the test set and gets 100% accuracy. But I think it is an honest mistake.
questions: Does g^t in line 153 represent G_beta in eq.11? Or it is a learnable parameter? 
Why beta=1 in eq.11 means single spike coding pattern of integration accumulation? In my opinion, when beta=1, G_beta = g^t * C, which behaves like (1 – tau) * C in the LIF neuron (V = (1-tau) * V + tau * C).

In line 167, “GLIF only performs hard reset on the exponential-decay-related part of the membrane potential”. Can the author explain why use such a hard reset? In most cases, hard reset means the whole V, rather than a part of V, is set to V_reset .
limitations: I find from codes that some parameters of GLIF are not reused across different time-steps because they have shape [T, *] where T is the total time-steps. I am afraid that this design violates the parameter reuse rule in SNNs.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: In this work, the authors propose a mixture type of neuron model called GLIF to introduce additional flexibility to the neurons. They demonstrate that the proposed model can achieve SOTA performances on a bunch of classification tasks. 
strengths_and_weaknesses: Strengths:
1. The paper is written clearly. 
2. The model achieved SOTA performances.
Weakness:
1. There could be more analyses on why the proposed method works beyond the comparison to other methods. 
questions: 1. How does the distribution of $\alpha$, $\beta$, and $\gamma$ look like? Does it display heterogeneity over layers, especially for the phenotypes defined in Figure 2?
2. Since each neuron here contains three branches, it makes me feel that its computational cost is actually equivalent to three times the regular one, where the advantage of the current approach is not as high as it appears to be. 
3. Considering the model complexity, it would also be interesting to compare with the model ensembled with three different configurations for a single neuron type in order to support the claim that the heterogeneity in the same model makes a difference. 
limitations: NA
review_title: null

Meta-review: metareview: Although the scores are boarderline, the reviewers appreciate the proposed model as a novel as an interesting generalization of multiple existing models. The authors ironed out a bug in the evaluation, that was caught by a review, and report that results still hold (on a larger dataset). 
confidence: Less certain
Decision: Accept