From: ICLR.cc/2023/Conference/Paper1631/Authors
title: Submission Withdrawn by the Authors

From: ICLR.cc/2023/Conference/Paper1631/Authors
title: Thank you very much for reviewing. A friendly reminder. 
comment: Dear Reviewer,


Thanks for your constructive feedback and suggestions. We gently remind you that the discussion period is coming to an end. We've tried to alleviate your concerns, particularly by clarifying some less clear descriptions that you mentioned. In addition, we've revised the manuscript to make it more clear. 

Please let us know if you have any additional concerns. 


Best regards, 

Authors.

From: ICLR.cc/2023/Conference/Paper1631/Authors
title: Thanks for your review. A Friendly Reminder.
comment: Dear Reviewer,



Thanks for your constructive feedback and suggestions. We gently remind you that the discussion period is coming to an end. We've tried to alleviate your concerns, particularly by clarifying some less clear descriptions that you mentioned. In addition, we've revised the manuscript to make it more clear. 

Please let us know if you have any additional concerns. 



Best regards, 

Authors.

From: ICLR.cc/2023/Conference/Paper1631/Reviewer_DsgC
title: Response
comment: Thanks for the clarifications but I can't help but disagree with some of the framing of the problem (e.g. this is not probabilistic inference in any meaningful way). While the text edits have helped somewhat I still feel that there are some pretty fundamental issues with the way the material is being presented that make it hard to appreciate the novelty or significance of the work (which in themselves I consider to be rather incremental). As such, I will not be changing my original overall score.

From: ICLR.cc/2023/Conference/Paper1631/Authors
title: Thanks for your reply!
comment: We greatly appreciate your approval. We will do our best to improve the manuscript and hope to present a better version. 

Regards.

Authors

From: ICLR.cc/2023/Conference/Paper1631/Authors
title: Thanks for your feedback! Our response is included below. 
comment: We sincerely appreciate your reply. 

We understand your concern about Noisy LIF, as adding a component to classical LIF can be perplexing. However, Noisy LIF is based on existing neuron models (with some modifications) [1,2,3]. These works used differential equations to model LIF with noise. They considered the diffusion approximation and described the current input to the neuron by a deterministic part and a random noise part, which leads to the additive noise term in the discrete form. Thus, rather than changing the classical LIF model and proposing a new neuron with a noisy sub-threshold dynamic, *we use another classical model (which is noisy itself) as the basic unit of the NSNN framework*. Also, as you can see, we made some modifications to establish the NSNN framework; for example, Noisy LIF uses probabilistic firing rather than Heaviside firing. This is a mechanism borrowed from the 'escape rate', i.e., the spike is emitted with a probability density $p=f(u-v_{th})$ [1].

The novelty of this work, as stated in the manuscript, lies in the NSNN framework rather than in the proposal of a new noisy neuron model. **Although the basic unit (noisy neurons) has a long history, no research, to our knowledge has proposed a framework capable of supporting large networks.**

We have made changes in the updated version to clarify this, and we hope that this will help avoid possible confusion. We hope we have addressed your doubts. Thanks again for your response!

Best regards.

**Refs**

[1] Plesser, Hans E., and Wulfram Gerstner. "Noise in integrate-and-fire neurons: from stochastic input to escape rates." *Neural computation* 12.2 (2000): 367-384.

[2] Burkitt, Anthony N. "A review of the integrate-and-fire neuron model: I. Homogeneous synaptic input." *Biological cybernetics* 95.1 (2006): 1-19.

[3] Patel, Ashok, and Bart Kosko. "Stochastic resonance in noisy spiking retinal and sensory neuron models." *Neural networks* 18.5-6 (2005): 467-478.

From: ICLR.cc/2023/Conference/Paper1631/Reviewer_dsY7
title: Response
comment: Thank you for responding to the comments.
I think my own brief comments were a subset of those of the other reviewers, who in turn had more substantial things to say.  I maintain that the fundamental contribution is sound, although the details raised by other reviewers are valid.

From: ICLR.cc/2023/Conference/Paper1631/Reviewer_38qu
title: Additional Comments
comment: Dear authors,

I feel that the second doubt was not fixed well. I focus on the usage of an internal randomness. When one proposed a new model or introduced a new component into the classical model, the detailed discussions and evaluation of its advantages is unavoidable. I would have appreciated more clarity on the benefits brought from the internal randomness, how to leverage the noise, etc.. 

Best Regards

From: ICLR.cc/2023/Conference/Paper1631/Authors
title: We appreciate you taking the time to comment! Please let us know if you still have questions and we are happy to answer or discuss them.
comment: Dear reviewer, 

We'd like to thank you for your insightful comments and constructive suggestions again. We have addressed each of your questions and added clarifications to our revised manuscript. We hope they can be helpful in addressing your concerns. Please let us know if you have any further questions,  and we'll be happy to answer or discuss them. 

Best regards.

From: ICLR.cc/2023/Conference/Paper1631/Authors
title: Thanks again for your comments!
comment: Dear reviewer, 

We’d like to express our gratitude for your detailed comments and constructive suggestions. We have responded to each of your questions and added clarifications to our revised manuscript. We hope they could be helpful addressing your concerns. In addition, we are more than happy to address any further questions before the conclusion of the rebuttal period.

Regards.

From: ICLR.cc/2023/Conference/Paper1631/Authors
title: Thank you for your comments! We have enhanced the manuscript!
comment: Dear reviewer,

Thanks again for your comments. Please let us know if you still have questions and we are happy to answer or discuss them.

Regards.

From: ICLR.cc/2023/Conference/Paper1631/Authors
title: Thank you for taking the time to review. Please let us know if you have any further questions.
comment: Dear reviewer,

We’d like to express our sincere gratitude for your detailed comments and constructive suggestions. We have responded to each of your questions and added clarifications to our revised manuscript, and we hope they can help address your concerns. Your insightful comments significantly improve the quality of this manuscript. 

Please let us know if you have any further questions, and we will be happy to answer or discuss them.

Best regards.

From: ICLR.cc/2023/Conference/Paper1631/Authors
title: Thank you very much for your feedback!
comment: # Response

Many thanks for your reply! We have read your comments carefully and replied separately as follows.

---

1. First, the potential performance benefit of introducing internal noise has been empirically and theoretically investigated in previous works, such as dropout [1], which introduces multiplicative noises to neurons. In [2], noise injection is proven to improve generalization and robustness with the neural SDE frame. Gaussian noise injection is an effective regularization technique, as demonstrated by [3]. [4] confirmed the robustness improvement brought by SDE-based RNN models. Noisy LIF can be obtained by discretizing an SDE with a specific drift and diffusion coefficient. In addition, the Brownian noise term is consistent with the fluctuation of membrane voltage in biological reality, which results in an additive gaussian noise term in the discrete form. 

   Second, "introducing internal noise leads to improvement of bio-realism." It is well-supported by prior research. Neuroscientists consider membrane noise an essential element of the sub-threshold dynamics of neurons. A neuron model with noise can reproduce the complex response properties of biological neurons [5,6,7]. 

   We did our best to make this clear in the limited space in the revised version, and we hope it can address your concerns!

---

2. Thank you very much for your detailed reply! 
   Regarding the formation in Eq.2: the variance of the epsilon used in the experiment is a "moderate" value (see Section 4.3 for details), so the input current continues to play a major role in the membrane voltage. The addition of epsilon introduces "some" randomness to the firing process and will not harm the performance.
   Section 4.1 demonstrated the generalization superiority of NSNN (learned by NDL) on DVS-CIFAR and DVS-Gesture datasets. Moreover, Section 4.2 and appendix A.2.2 show the advantage of the NDL-learnt NSNNs in terms of robustness! As demonstrated by the neural representation (neural code, presented by raster plots) in Section 4.4 Figure 5.A, NSNNs can construct a more fault-tolerant representation space. Although the two neural codes have apparent differences, they produce remarkably similar predictions. This part is consistent with the experiments in A.2.2, where NSNN maintains considerable resilience even facing very challenging hidden state-level perturbations (perturb spike outputs of all neurons).
   Thus, we show that the intrinsic randomness of NSNN is helpful against external perturbations, a result consistent with [2,3,4].

---

3. In Eq.5, the $\mathbf{o}^t\_l$ includes all spike outputs $o^t\_{l,m}$ . By Eq 3,4, the probability of $o^t\_{l,m}=1$ is equal to $\mathbb{P}[o^t\_{l,m}=1]=CDF_\epsilon (u^t\_{l,m} - v\_{th})$. These $o^t\_{l,m}$ are drawn from their corresponding distribution $Ber\big( \mathbb{P}[o^t_{l,m}]=1 \big)$ , so they obey the Bernoulli distribution. 

---

4. Fixed in the updated version.

---

Lastly, thank you for your detailed feedback; your thoughtful comments significantly improved the quality of this manuscript!

Sincerely,

Authors. 

## Refs

[1] Geoffrey E Hinton, et al. Improving neural networks by preventing co-adaptation of feature detectors. *arXiv*, 2012.

[2] Liu, Xuanqing, et al. How does noise help robustness? explanation and exploration under the neural sde framework. In *CVPR*. 2020.

[3] Camuto, Alexander, et al. Explicit regularisation in gaussian noise injections. In *Advances in Neural Information Processing Systems*, 2020.

[4] Soon Hoe Lim, N Benjamin Erichson, Liam Hodgkinson, and Michael W Mahoney. Noisy recurrent neural networks. In *Advances in Neural Information Processing Systems*, 2021.

[5] Kempter, R., Gerstner, W., Van Hemmen, J. L., & Wagner, H. (1998). Extracting oscillations: Neuronal coincidence detection with noisy periodic spike input. *Neural computation*, *10*(8), 1987-2017.

[6] Jacobson, Gilad A., et al. Subthreshold voltage noise of rat neocortical pyramidal neurones. *The Journal of physiology* 564.1 (2005): 145-160.

[7] DeFelice, Louis J. *Introduction to membrane noise*. 2012.

From: ICLR.cc/2023/Conference/Paper1631/Reviewer_38qu
title: Additional Comments
comment: I have read the other reviewer's comments and the authors' rebuttal, and appreciate the authors for their careful responses. There are obvious improvements between the original and revised versions of this paper, including a clarification on the motivations of introducing randomness into SNNs and the re-formulation of basic equations of the noisy SNNs. However, there are still some doubts.

1. The twofold motivations proposed in Section Introduction seem to be perfunctory. As claimed, introducing noise into SNNs not only promotes the generalization performance of SNNs, but also be more biologically realistic. It is better to discuss the motivations in-depth. 

2. I still have doubts on the formation of Eq. (2). As introduced, $\epsilon$ belongs to the Gaussian distribution. It looks like an internal randomness inside the LIF spiking neuron. I have to admit that exploiting Gaussian noise, for convenience, coincides with Bayesian networks. It seems to be counterintuitive that adding internal noise improves the performance of SNNs, since  we usually ensure the robustness of learning models against external noise, e.g., the noise inside the neural encoding.

3. Furthermore, when one uses the Gaussian internal noise, how to ensure the output to obey the Bernoulli distribution, as shown in Eq.(5)?

4. There are still too many mistakes in the formulation and computation process of NSNNs. For instance, what is the subscript "\ l, m" in Eq.(8)? These minor issues may heavily hinder the quality of this paper.

From: ICLR.cc/2023/Conference/Paper1631/Authors
title: Response to dsY7
comment: # Response to dsY7

Many thanks for your reviews! We are very grateful for your kind appreciation of our work. We hope that we have addressed your comments in our response below.

**Reviewer comment:** "The manuscript introduces a noisy interpretation of the standard LIF neuron, and shows that it can lead to probabilistic inference. Notably, the surrogate gradient technique is shown to be a "special case" of this, meaning that SG can be interpreted probabilistically."

**Response:**  Thank you for your appreciation! 

Our NSNN provides a general theoretical framework for SNNs, including the common deterministic ones (if  $Var[\epsilon]\rightarrow 0$ , its CDF approaches the Heaviside function and the probabilistic firing $o \sim P[o=1]=CDF_\epsilon(u-v_{th})$ becomes the Heaviside firing)! We hope that SNN research will benefit from the NSNN framework and NDL!

------

**Reviewer comment:** "there is too much information in the the manuscript meaning that the figures are too compact."

**Response:** Thank you for pointing this, we have amended the paper to address this! 

------

**Reviewer comment:** "SG is the leading means of training spiking nets, but is quite ad-hoc. The manuscript explains SG in terms of a probabilistic approach. That is non-trivial and useful."

**Response:** Thanks for your appreciation. As you mentioned, the commonly-used surrogate gradient tech is quite ad-hoc. We hope our work can bring understanding to the SNN field and beneficial future research.

From: ICLR.cc/2023/Conference/Paper1631/Authors
title: Part 2/2
comment: # Part 2/2

**Reviewer comment:** "Some expressions in this paper are very vague. For instance, there have been a lot of "topologies" defined in SNNs, such as the dynamics in [1] and dual/balance in [2]. What does the "topology" of SNNs in this work mean? "

**Response:** Thank you for mentioning these two exciting works [6,7]! Apologies for the confusing wording. We have amended this in the updated paper to avoid potential misunderstanding!

------

**Reviewer comment:** "The formulation of the LIF equation in Eq. (1) is inapposite since Eq. (1) does not we usually avoid such an ambiguity in the SNN community, although this community, although this formation leads to a more intuitionistic operation."

**Response:** We apologize for the lack of clarity in the previous presentation; we had hoped that using the form in Eq.1 (old version) would make the subsequent discrete process more readable.

Although the discrete neuron model we use is derived from the SDE (with a specific drift and diffusion), the meaning of *the expression is clear enough on its own* (membrane potential-level random noise injection). Therefore we have simplified and reorganized related content. We hope that the revised manuscript can address your concerns.

------



[1] Sietsma, Jocelyn, and Robert JF Dow. "Creating artificial neural networks that generalize." *Neural networks*.

[2] Hinton, Geoffrey E., et al. "Improving neural networks by preventing co-adaptation of feature detectors." *arXiv preprint arXiv:1207.0580* (2012)

[3] Soon Hoe Lim, N Benjamin Erichson, Liam Hodgkinson, and Michael W Mahoney. Noisy recurrent neural networks. In Conference on Neural Information Processing Systems (NeurIPS), 2021.

[4] Faisal, A. Aldo, Luc PJ Selen, and Daniel M. Wolpert. "Noise in the nervous system." *Nature reviews neuroscience* 9.4 (2008): 292-303.

[5] Rudolph, Michael, and Alain Destexhe. "Characterization of subthreshold voltage fluctuations in neuronal membranes." *Neural Computation* 15.11 (2003): 2577-2618.

[6] Shao-Qun Zhang, Zhao-Yu Zhang, and Zhi-Hua Zhou. Bifurcation Spiking Neural Network. Journal of Machine Learning Research (JMLR), 22(253):1-21. 2021.  

[7] C.-N. Chou, K.-M. Chung, and C.-J. Lu. On the algorithmic power of spiking neural networks. In Proceedings of the 10th Innovations in Theoretical Computer Science Conference, pages 26:1–26:20, 2019.

[8] Maass, Wolfgang. "Noise as a resource for computation and learning in networks of spiking neurons." *Proceedings of the IEEE* 102.5 (2014): 860-880.

From: ICLR.cc/2023/Conference/Paper1631/Authors
title: Response to 38qu  Part 1/2
comment: # Response to 38qu  Part 1/2

Many thanks for your review! Your criticisms have been taken onboard in the updated version. We hope that we have addressed your comments in our response below.

------

**Reviewer comment:** "There is a need for clarifying the motivations of this paper. For example, why should the authors introduce the noise into the LIF equation, that is, Eq. (2)? When the randomness is mentioned in machine learning, one tends to consider the randomness or noisy data rather than introducing the uncertainty into a deterministic model. What are the benefits of this way? The current version looks like ponderously using a mathematical tool on the SNN."

**Response:** The advantage of introducing internal noise into the SNN model is twofold. First, it leads to generalization improvements [1,2,3] (proved by Section 4.1-4.2 in the paper). Second, by introducing internal randomness, we improve the biological realism of the dynamics [4,5] of the spiking neuron model and also reproduce some phenomena exhibited in neural coding in biological neurons (Section 4.4). 

The intriduction of internal noise, making NSNN a *general theoretical framework for SNNs*! The Noisy LIF neuron combines the stochastic dynamics of the membrane potential (discretization of a specific SDE) and the stochastic firing process-the expression of firing probability- $P[\text{firing}]=P[u+\epsilon > v\_{th}]\overset{\text{as }\epsilon=-\epsilon}{=}P[\epsilon < u-v\_{th}]\overset{\text{by def. of CDF}}{=}\operatorname{CDF}_\epsilon(u-v\_{th})$ naturally raises a probabilistic firing mechanism (Eq. 3-4).  If the $\epsilon$'s variance approaches zero (declining randomness), its CDF will approach a Heaviside function; hence, the Noisy LIF here actually covers the case of a deterministic LIF neuron (threshold firing)! Further, if we consider an epsilon from a logistic distribution, the Noisy LIF describes the case of sigmoidal neurons [8]! Actually, other continuous noises whose mean is zero and satisfy $\epsilon=-\epsilon$ can also be used! 

The introduction of the internal noise allows us to measure the firing sensitivity by statistics of membrane noise (important to derive the noise-driven learning). Using Noisy LIFs, we build the NSNN; although each neuron in the network produces simple spike outputs, the mixture over them can approximate complex target distributions, leading to a powerful stochastic recognition machine learning model (proved by Section 4.1-4.2). 

Finally, thank you very much for your comments!  They have been taken onboard and have greatly impreoved the manuscript!

From: ICLR.cc/2023/Conference/Paper1631/Authors
title: Part 2/2
comment: # Part 2/2

**Reviewer comment:** "would have liked a little more discussion on the effects of overall noise level on performance and the procedures used for setting that hyperparameter"

**Response:** Thanks for your comments. Section 4.3 of the revised paper now includes the relevant experiment and discussions!

------

**Reviewer comment:** "the definition of the noisy LIF is unnecessarily convoluted for such a simple concept,In particular I found the way the relationship between membrane potential and spiking is described as completely confusing. It should simply say that there is a nonlinear mapping between deviations from rest and probability spiking via a bernoulli noise model, but the notation is bizarre and the introduction of the nonlinearity unnecessarily backwards."

"i found table 1 notation very hard to read", "many grammar issues "

**Response:** Thank you for pointing out the problems. In response to your comments, we have made changes and amendments to the corresponding content in the paper!

*relation ship between membrane potential and spiking:* Introducing $\epsilon$, which turns membrane potential into a random variable, allows us to establish the relationship between membrane potential noise and neuron firing using $P[\text{firing}, i.e., o=1]=P[u+\epsilon > v\_{th}]\overset{\text{as }\epsilon=-\epsilon}{=}P[\epsilon < u-v\_{th}]\overset{\text{by def. of CDF}}{=}\operatorname{CDF}_\epsilon(u-v\_{th})$. As a result, the stochastic firing mechanism is naturally raised (in Eq.3, 4 of the revised paper), and the "neuron encodes a random variable" principle is implemented. 

*It makes NSNN a general theoretical framework for SNNs*!  If the $\epsilon$'s variance approaches zero (declining randomness), its CDF will approach a Heaviside function; hence, the Noisy LIF here actually covers the case of a deterministic LIF neuron (Heaviside firing). Further, if we consider an epsilon from a logistic distribution, the Noisy LIF describes the case of sigmoidal neurons [8]! Actually, you can use other noises, whose mean is zero and satisfy $\epsilon=-\epsilon$. 

------

**Reviewer comment:** "title is misleading, since there is no probabilistic inference involved in the tasks presented, just averaging of internal noise"

**Response:** NSNNs perform probabilistic inference on Bayesian networks with spiking neurons by sampling. By saying probabilistic inference, the typical computation that needs to be carried out can be expressed as $p(\mathbf{o}\_L^t | x) = \sum_{\text{all possible values}} p(\mathbf{o}\_{1\cdots L}^t | x)$, which is exactly what we presented in Eq.6. 

"just internal noise averaging" We must emphasize that the inference is not an averaging of internal noise. The NSNN inference is not running the same net $T$ times and then averaging its output. The distribution of the random variable (spike state) represented by each neuron $p(o\_{l,m}^t | \operatorname{pa}(o\_{l,m}^t))$ is conditioned on its spatial and temporal parent nodes. Thus the network's joint distribution $p\_\theta(\mathbf{o}\_{1\cdots L}^t)$ varies over time. We understand your confusion as the tasks we considered are classification problems whose target distribution is time-invariant. And as a supplement, we have included a toy time series forecasting problem in Section 4.4. 

------

## Refs

[1] Neal, Radford M. "Connectionist learning of belief networks." *Artificial intelligence* 56.1 (1992): 71-113.

[2] Tang, Charlie, and Russ R. Salakhutdinov. "Learning stochastic feedforward neural networks." *Advances in Neural Information Processing Systems* 26 (2013).

[3] Raiko, Tapani, et al. "Techniques for learning binary stochastic feedforward neural networks." *ICLR* (2015).

[4] Averbeck, Bruno B., Peter E. Latham, and Alexandre Pouget. "Neural correlations, population coding and computation." *Nature reviews neuroscience* 7.5 (2006): 358-366.

[5] Bengio, Yoshua, Aaron Courville, and Pascal Vincent. "Representation learning: A review and new perspectives." *IEEE transactions on pattern analysis and machine intelligence* 35.8 (2013): 1798-1828.

From: ICLR.cc/2023/Conference/Paper1631/Authors
title: # Response to DsgC Part 1/2
comment: # Response to DsgC Part 1/2

We thank the reviewer for thoughtful comments! Your criticisms have been taken onboard in the updated paper. We respond to your points below, and we hope that it is now clear what our work does and it is novel.

------

**Reviewer comment:** "the logic of combining an explicit probabilistic model of the input output function with the loss is weird, and the framing of the problem conceptually muddled. As I understand it, the network implements a stochastic recognition model and not a generative process. The local loss definition is weirdly paired with the classical supervised loss. What should happen is write out the global loss, which now includes an expectation over the internal noise of the recognition model then derive local gradients for that, which themselves will involve expectations over the noise, which then get replaced with empirical averaging. "

**Response:** We apologize for the lack of clarity in our presentation. To address your concerns, we have rearranged and amended the related contents in the revised manuscript. We hope that the revised version can address your concerns. The formulation is not weird [1,2,3], and the loss in Eq. 5 is an "instantaneous" loss,  thus leads to expression $\nabla\_\theta \mathbb{E}\_{\mathbf{o}} p\_\theta (\mathbf{o}) f$  (Eq. 7).

------

**Reviewer comment:** "at the end of it all, the expression of the parameter learning is equivalent to reinforce so i' not entirely sure what has been gained in the process of doing all this. It looks like one does proper backprob most of the way and reinforce at the end step. It is not clear what if anything does the added mathematical complexity bring to the table. "

**Response:**  *NDL is not reinforce*. We have rearranged the derivation in the updated paper, which we hope will clear up misunderstandings. The only relationship we realize between NDL and reinforce appears to be confined to Eq. 9, where the estimator relates to a reinforce estimator with a fully-informed baseline, but this is just a transitional step in our derivation. To obtain NDL in Eq. 13, we employ a first-order approximation paired with a corollary of Proposition 1. 

As you mentioned, the actual computation in the backward pass appears to be fairly trivial, but this is exactly where NDL's elegance comes in. It is derived rigorously step by step. In particular, the 1-order approximation we use on Eq. 9 brings the very friendly term $\partial L/\partial o$ (but also introduces bias to the estimator), so we do not need to compute an additional gradient generator in the forward pass. Also, the form of Eq. 13 establishes a probabilistic explanation by mathematical links to the ad-hoc surrogate gradient learning (Reviewer dsY7).

------

**Reviewer comment:** "technical concerns about spiking nets training for ML purposes are interspersed with biological commentary in a way that makes me slightly confused about the target audience for this work; why focus on the nature of the code and fano factors etc, when the performance of the trained models is assessed using the noise free network?"

**Response:**  Thank you for your comments, we have revised and reorganized the corresponding contents in the updated version!

"why focus on the nature of the code and fano factors etc, when the performance of the trained models is assessed using the noise free network?" Neural encoding is a central topic in computational neuroscience [4], and the machine learning community is interested in representation learning [5]. For spiking networks, the neural code corresponds to the representation in ANNs. 

"...assessed using the noise free network." We clarify that, *rather than using the noise-free network, we use unperturbed inputs to ensure the observation information is identical for each trial*, i.e., the stimuli are identical for each trial. NSNN models are inherently stochastic (resulting in trial-to-trial variability), which is a prerequisite for most neural code analysis techniques. Our neural code analysis using NSNN demonstrates its potential as a tool for computational neuroscience research rather than just a functional machine learning model. With NSNN, we hope to narrow the gap between machine learning models and the coding frameworks used in computational neuroscience and to offer understanding for both sides.

From: ICLR.cc/2023/Conference/Paper1631/Authors
title: part 3/3
comment: # part 3/3



**Reviewer comment:**"Eq.9: there is a big jump in the math derivation from Eq. 8 to Eq. 9 and I cannot follow that."

**Response:** We have amended this in the updated version to describe the derivation more clearly! Eq.8-9 are marked as Eq. 11-13 in the revised paper. To be brief, the NDL is derived from $\sum_m \nabla_{\theta_l} p_{\theta_l} (o_{l,m}^t|\operatorname{pa}(o_{l,m}^t)) (L-L^\prime)$, where $\operatorname{pa}$  denotes parent nodes. By computing $\nabla_{\theta_l} p_{\theta_l} (o_{l,m}^t|\operatorname{pa}(o_{l,m}^t))$ using proposition.1 and approximate $(L-L^\prime)$ , we get the NDL in Eq.13.  

------

**Reviewer comment:** "I am puzzled that why the Fano factor in Fig. 3B is very small since the neuronal model generates stochastic spikes based on a Bernoulli process. Considering the membrane potential noise, the Fano factor should be larger than one."

**Response:** The very small FF is caused by a bug in our plotting codes' axis labeling part (it changes the order of magnitude of the coordinates). We have fixed it, redrawn all corresponding figures, and confirmed that the original conclusion remains unchanged. Similarly, we see no FF larger than 1 in the updated Fig. 5.B. We offer two possible reasons for this. First, since the tasks considered in this part are all classification, their target distributions are time-invariant, which may reduce the randomness of the neural code in the well-trained networks. Second, the tasks considered in this part have short simulation timesteps (2 or 10). As a complement, we present additional experiments for a classification task and a regression task with longer timestep in the updated Section 4.4 and observed some FF>1 cases.

------

**Reviewer comment:**"Eq. 11: SG is undefined."

**Response:** Fixed in the revised version (new Eq. 14)! 

----

**Reviewer comment:** the single-neuron model is not new (Eqs. 1-3) and the distribution of neuronal spiking responses (the math on the page 4) is also based on previous studies. Perhaps the novel result is the derived learning rule of the stochastic spiking neurons (Eq. 10).

**Response:** As stated in the manuscript, the novelty of this work lies in the NSNN framework rather than in the proposal of a new noisy neuron model (Noisy LIF is based on earlier neuron models). 

Internal noisiness or randomness of a cell does not account for all the variability in spike trains in vivo [8] since a neuron is both an individual and a component of a neuron network. However, a well-defined framework for the networks of noisy spiking neurons is missing. This is the meaning of the proposed NSNN framework, in which we provide scalable and easy-to-implement inference and corresponding learning rules for networks of noisy spiking neurons. 

Also, we highlight the additional novelty of noise-driven learning; at the microscopic (neuronal) level, NDL offers a meaningful interpretation of the surrogate gradient (an ad-hoc solution for training SNNs, Reviewer dsY7). Theory is often about demonstrating that two seemingly unrelated things are, in fact, the same thing / related things but in two different lights. We have shown this for noise-driven learning and surrogate gradients within the NSNN frame. 


## Refs

[1] Fiete, Ila R., and H. Sebastian Seung. "Gradient learning in spiking neural networks by dynamic perturbation of conductances." *Physical review letters* 97.4 (2006): 048104.

[2] Zhang, Wenhao, et al. "Sampling-based Bayesian inference in recurrent circuits of stochastic spiking neurons." *bioRxiv* (2022).

[3] Ma, Wei Ji, et al. "Bayesian inference with probabilistic population codes." *Nature neuroscience* 9.11 (2006): 1432-1438.

[4] Vasudeva Raju, Rajkumar, and Zachary Pitkow. "Inference by reparameterization in neural population codes." *Advances in Neural Information Processing Systems* 29 (2016).

[5] Maass, Wolfgang. "Noise as a resource for computation and learning in networks of spiking neurons." *Proceedings of the IEEE* 102.5 (2014): 860-880.

[6] Plesser, Hans E., and Wulfram Gerstner. "Noise in integrate-and-fire neurons: from stochastic input to escape rates." *Neural computation* 12.2 (2000): 367-384.

[7] Burkitt, Anthony N. "A review of the integrate-and-fire neuron model: I. Homogeneous synaptic input." *Biological cybernetics* 95.1 (2006): 1-19.

[8] Gerstner, Wulfram, et al. *Neuronal dynamics: From single neurons to networks and models of cognition*. Cambridge University Press, 2014.

From: ICLR.cc/2023/Conference/Paper1631/Authors
title: part 2/3
comment: # part 2/3

**Reviewer comment:** "Doubly stochastic neuron I realize the single-neuron model is doubly stochastic: It has a Gaussian white noise in its membrane potential dynamics (Eq.2) as well as the stochastic spiking emission (the bottom of page 3). It is better to add the motivation of considering the doubly stochastic neurons, and/or whether the whole framework works if we remove the Gaussian white noise in membrane potential dynamics. Also, the probability in the Bernoulli distribution at the bottom of page 3 is not defined and I need to guess its derivation from some text. "

**Response:** The noisy LIF we introduced is based on previous works [6,7], with some modifications in order to build the NSNN framework. As you mentioned, the Noisy LIF incorporates membrane potential noise $\epsilon$ and a random firing mechanism. The two are actually *closely linked* and allow the NSNN theoretical framework to be built! 

Introducing membrane noise, which turns membrane potential into a random variable, allows us to establish the relationship between membrane potential noise and neuron firing using $P[\text{firing}, i.e., o=1]=P[u+\epsilon > v\_{th}]\overset{\text{as }\epsilon=-\epsilon}{=}P[\epsilon < u-v\_{th}]\overset{\text{by def. of CDF}}{=}\operatorname{CDF}_\epsilon(u-v\_{th})$. As a result, the stochastic firing mechanism is naturally raised (in Eq.3, 4 of the updated paper), and the "neuron encodes a random variable" principle [5] is implemented. 

*The Noisy LIF makes NSNN a general framework for SNNs.*  If the $\epsilon$'s variance approaches zero (declining randomness), $CDF_\epsilon$ will approach a Heaviside function; hence, the Noisy LIF here actually covers the case of a deterministic LIF neuron (Heaviside firing). Further, if we consider an epsilon from a logistic distribution, the Noisy LIF describes the case of sigmoidal neurons [5]. 

"Remove the membrane potential noise" It corresponds to Sec. 4.3 in the revised paper, when the variance of $\epsilon$ tends to zero, the probabilistic firing will approach a Heaviside firing (as in deterministic SNNs) since $o\sim Ber(P[o=1])=Ber(CDF\_\epsilon (u-v\_{th}))=Ber(Heaviside(u-v_{th}))$. NDL computes the post-synaptic factor (measuring the firing sensitivity) using the PDF of membrane potential noise, it will not be able to calculate the neuron's firing sensitivity if epsilon is removed (refer to Sec. 4.3) ; hence the name **noise-driven** learning.

We revised and improved the content of the corresponding section in response to your feedback. We hope that our response and revision can address your concern.

------

**Reviewer comment:** "dW in Eq. 2: it seems that dW is a derivative of Wiener process rather than a Levy process as described below Eq. 2."

"scaling factor \delta/tau* below Eq. 3: strictly speaking, the scaling factor \delta/\tau* cannot be incorporated in weight parameter \theta. That is, the weight \theta should be fixed and is irrelevant with the the \delta which is only a parameter in simulating the network.its dynamics"

**Response:** Thanks for pointing this! We note that although many models use "absorbing the $\Delta t$ into $\theta$" (e.g., in iterative SNNs, or in residual networks, which are regarded as the discretization of neural ODEs), strictly, this is a flawed description since the simulation timestep $\Delta t$ should be untied with the model parameters. 

We have addressed these issues in the revised manuscript. Given that the majority of the paper is concerned with the discrete neuron model, we have streamlined and amended corresponding descriptions. We hope that our response and revision can address your concern.

From: ICLR.cc/2023/Conference/Paper1631/Authors
title: Response to pjQ1  part 1/3
comment: # Response to pjQ1  part 1/3

Thank your very much for your review! We are grateful for your kind apprecation. We hope that our response below addresses your concerns! 

------

**Reviewer comment:** "The first-order approximation in Eq. 8. I doubt whether the 1st order approximation works at here because the output o is binary. It is unclear how much the derived learning rule is dependent on this approximation, and what the learning rule looks like without using this approximation. Or the derivative over o should be only viewed as a math notation. "

**Response:** This approximation works. The first-order approximation here is used mainly to address the scalability issue encountered when using Eq.10 for NSNN learning. Directly adopting Eq.10 incurs heavy computation as we must evaluate the final loss value after each neuron's state change, which is unwise when the network becomes larger. Inspired by some previous works [1], we propose to attribute the loss value change brought by one neuron's spike state change to this neuron. So you can think of it as a mathematical notation. A step further, we use a first-order approximation to approximate the original loss value difference term in Eq.10. The approximation step is important in NDL since it brings a common $\partial L/\partial o$ term (representing the global learning signal) and a $(2o^t_{l,m}-1)$ term. The $(2o^t_{l,m}-1)$ term is then combined with another obtained using Proposition 1 (as $(2o-1)^2\equiv 1$), giving us the NDL rule in a very succinct mathematical form (Eq. 13). NDL is novel and offers insights for surrogate gradient as well-" The manuscript explains SG in terms of a probabilistic approach. That is non-trivial and useful" (from reviewer dsY7)! 

---



**Reviewer comment:**"A conceptual concern of neural coding mechanism in the network model is that the math derivation still consider the discrete spiking response is an approximation of a continuous rate (such as the approximation used in Eq. 8 and the text descriptin in somewhere else in the paper). In this scenario, therefore there must be some accuracy that eventually impairs the performance of the network compared with the rate-based network with continuous output. Although I know this is quite conventional in the field of spiking neural network, I am quite interested in seeing some discussions about other possibilities for representing a continuous variable in a discrete spiking network. From computational neuroscience research, one possibility is that we consider a large neuronal population whose firing rate has a smooth profile over the encoded variable, and then the spiking response can encode a continuous distribution whose type is determined by the smooth firing rate profile."

**Response:** Thank you for bringing up these exciting works[2,3,4]! It is an interesting question how the information is encoded in a network of spiking neurons. As the experiments in the old version focused on classification tasks with time-invariant target distributions (more likely to result in a rate-based coding strategy), the new Section 4.4 expands on the neural coding analysis with a toy regression task and shows correlation-based coding possibilities. By analyzing neural codes in NSNNs, we demonstrate the NSNN's potential as a tool for neural coding analysis, which we hope will narrow the gap between machine learning models and computational neuroscience coding analysis frameworks!

From: ICLR.cc/2023/Conference/Paper1631/Authors
title: Paper update
comment: Dear reviewer and chairs, 

We appreciate all reviewers for their valuable feedback! We have made significant efforts to address their comments, questions, and concerns. We believe it has elevated the quality of the paper! 

Major revisions (details can be found in individual responses to the reviewers below) are summarized as follows: 
(1) The paper was reorganized to make it more clear. In particular, an experiment was postponed to the appendices to enable us to gain space and make the paper easier to read. 
(2) We modified the 'introduction' section to make our claims more clear. We added references that suggested by the reviewers. 
(3) The formulation and derivation were more clearly stated in Section 3. 
(4) Based on the reviewers' feedback, we have added and expanded some content in Section 4. We added in Section 4.3 a study about the effect of NSNN internal noise level on performance. We have expanded the discussion of neural code in Section 4.4.

Regards.

From: ICLR.cc/2023/Conference/Paper1631/Reviewer_pjQ1
confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
summary_of_the_paper: The paper proposes a stochastic spiking network and normatively derives a three-factor learning rule. Numerical experiments suggest some competitive performance of the model.
strength_and_weaknesses: Most of the theoretical derivations look good but few of them are flawed or need careful justification. The figures are nice, and the performance of the network was compared with other models.

### Major concern:
- The first-order approximation in Eq. 8.
I doubt whether the 1st order approximation works at here because the output o is binary.  It is unclear how much the derived learning rule is dependent on this approximation, and what the learning rule looks like without using this approximation. Or the derivative over o should be only viewed as a math notation. 

- A conceptual concern of neural coding mechanism in the network model is that the math derivation still consider the discrete spiking response is an approximation of a continuous rate (such as the approximation used in Eq. 8 and the text descriptin in somewhere else in the paper). In this scenario, therefore there must be some accuracy that eventually impairs the performance of the network compared with the rate-based network with continuous output. Although I know this is quite conventional in the field of spiking neural network, I am quite interested in seeing some discussions about other possibilities for representing a continuous variable in a discrete spiking network. From computational neuroscience research, one possibility is that we consider a large neuronal population whose firing rate has a smooth profile over the encoded variable, and then the spiking response can encode a continuous distribution whose type is determined by the smooth firing rate profile (e.g., Ma, Nat. Neurosci, 2006; Raju, NeurIPS 2016; Zhang, bioRxiv, 2022).

- Doubly stochastic neuron
I realize the single-neuron model is doubly stochastic: It has a Gaussian white noise in its membrane potential dynamics (Eq.2) as well as the stochastic spiking emission (the bottom of page 3). It is better to add the motivation of considering the doubly stochastic neurons, and/or whether the whole framework works if we remove the Gaussian white noise in membrane potential dynamics. Also, the probability in the Bernoulli distribution at the bottom of page 3 is not defined and I need to guess its derivation from some text. 

- dW in Eq. 2: it seems that dW is a derivative of Wiener process rather than a Levy process as described below Eq. 2.

- scaling factor \delta/tau* below Eq. 3: strictly speaking, the scaling factor \delta/\tau* cannot be incorporated in weight parameter \theta. That is, the weight \theta should be fixed and is irrelevant with the the \delta which is only a parameter in simulating the network.its dynamics

### Minor
- Eq.9: there is a big jump in the math derivation from Eq. 8 to Eq. 9 and I cannot follow that.

- I am puzzled that why the Fano factor in Fig. 3B is very small since the neuronal model generates stochastic spikes based on a Bernoulli process. Considering the membrane potential noise, the Fano factor should be larger than one.

- Eq. 11: SG is undefined.
clarity,_quality,_novelty_and_reproducibility: - Novelty
I am not in the field of using spiking networks to achieve benchmark tasks, and thus I am not very clear about the novelty of this study. What I can tell is that the single-neuron model is not new (Eqs. 1-3) and the distribution of neuronal spiking responses (the math on the page 4) is also based on previous studies. Perhaps the novel result is the derived learning rule of the stochastic spiking neurons (Eq. 10).

- Clarity: the paper is structure-wise and well-written overall. Figures and tables are easy to understand.

- Reproducibility: The paper presents most of the math derivations and the author also uploaded the code. I believe the paper can be reproduced although I haven't run and checked the details of the codes.
summary_of_the_review: I would recommend this paper if the derived learning rule is novel (need confirmation from authors). But I suggest that the authors could explain the bold 1st order approximation in the math derivation and add some discussions of encoding continuous variables in discrete spiking responses.
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
recommendation: 5: marginally below the acceptance threshold

From: ICLR.cc/2023/Conference/Paper1631/Reviewer_DsgC
confidence: 5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.
summary_of_the_paper: The paper expands standard formalisms for stochastic spiking neurons with leaky integrate ad fire dynamics (LIF) to define updates for parameter optimization. The new optimizer is shown to improve over a deterministic counterpart in classic benchmark classification taks, probably by smoothening the loss landscape and thus improving generalizability. It also seems more robust to adversarial attacks relative to deterministic spiking nets.
strength_and_weaknesses: + Strength:  attempt at scalable large scale stochastic gradient computation with noisy LIFs
- Weakness: the logic of combining an explicit probabilistic model of the input output function with the loss is weird, and the framing of the problem conceptually muddled. As I understand it, the network implements a stochastic recognition model and not a generative process. The local loss definition is weirdly paired with the classical supervised loss. What should happen is write out the global loss, which now includes an expectation over the internal noise of the recognition model then derive local gradients for that, which themselves will involve expectations over the noise, which then get replaced with empirical averaging. 
- W: at the end of it all, the expression of the parameter learning is equivalent to reinforce so i' not entirely sure what has been gained in the process of doing all this. It looks like one does proper backprob most of the way and reinforce at the end step. It is not clear what if anything does the added mathematical complexity bring to the table. 
- W: technical concerns about spiking nets training for ML purposes are interspersed with biological commentary in a way that makes me slightly confused about the target audience for this work; why focus on the nature of the code and fano factors etc, when the performance of the trained models is assessed using the noise free network?
- W: would have liked a little more discussion on the effects of overall noise level on performance and the procedures used for setting that hyperparameter

clarity,_quality,_novelty_and_reproducibility: - the definition of the noisy LIF is unnecessarily convoluted for such a simple concept,In particular I found the way the relationship between membrane potential and spiking is described as completely confusing. It should simply say that there is a nonlinear mapping between deviations from rest and probability spiking via a bernoulli noise model, but the notation is bizarre and the introduction of the nonlinearity unnecessarily backwards.
- the dimensionality of some of the gradients was hard to follow, given that the neuron and parameter indices tend to be dropped 'for simplicity'. I would find a clear vectorial notation whenever possible, and full indices otherwise much easier to follow.
- i found table 1 notation very hard to read
- many grammar issues 
- title is misleading, since there is no probabilistic inference involved in the tasks presented, just averaging of internal noise
summary_of_the_review: Overall, while numerical results are somewhat interesting, the text describing the mathematical framing of the problem is confusing and in places obfuscating, while the actual dynamics and learning procedures are simple and closely tight to previous results in the literature. As such, it is not entirely clear what conceptual benefit would the reader get from the manuscript in its current form. There is an implicit promise in providing a cleaner mathematical justification for surrogate gradients but the conceptual muddle of the optimization formulation does not really help that case. The numerical results seem solid but could benefit from a little expansion. The neural coding implications feel out of place for the overall paper format and framing but potentially interesting for computational neuroscientists wanting to understand its effects. 

Score-wise this is really a 4: not a hopeless case but would require very significant reframing of the problem and corresponding rewriting before I'd deem it publishable. 
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
recommendation: 3: reject, not good enough

From: ICLR.cc/2023/Conference/Paper1631/Reviewer_38qu
confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
summary_of_the_paper: This paper considers introducing randomness into spiking neural network, in which the noisy spiking neural network here is converted a Bayesian network. The experiments conducted on CIFAR exhibit the high robustness of the proposed method.
strength_and_weaknesses: Strengths:
1. This paper introduces a series of high-level tools, such as Ito SDEs, to reformulate SNNs with LIF neurons, from which the concern NSNN can be converted into a Bayesian network. The inference of this Bayesian network seems solid.
2. The experiments conducted on CIFAR exhibit the high effectiveness of the proposed method.

Weaknesses:
1. There is a need for clarifying the motivations of this paper. For example, why should the authors introduce the noise into the LIF equation, that is, Eq. (2)? When the randomness is mentioned in machine learning, one tends to consider the randomness or noisy data rather than introducing the uncertainty into a deterministic model. What are the benefits of this way? The current version looks like ponderously using a mathematical tool on the SNN.
2. Some expressions in this paper are very vague. For instance, there have been a lot of "topologies" defined in SNNs, such as the dynamics in [1] and dual/balance in [2]. What does the "topology" of SNNs in this work mean? 
3. The formulation of the LIF equation in Eq. (1) is inapposite since Eq. (1) does not avoid such an ambiguity in the SNN community, although this formation leads to a more intuitionistic operation.

[1] Shao-Qun Zhang, Zhao-Yu Zhang, and Zhi-Hua Zhou. Bifurcation Spiking Neural Network. Journal of Machine Learning Research (JMLR), 22(253):1-21. 2021.
[2] C.-N. Chou, K.-M. Chung, and C.-J. Lu. On the algorithmic power of spiking neural networks. In Proceedings of the 10th Innovations in Theoretical Computer Science Conference, pages 26:1–26:20, 2019.

clarity,_quality,_novelty_and_reproducibility: mentioned above.
summary_of_the_review: Overall, I believe this paper focuses on an interesting topic; however, the motivations still require the clear clarification. If these issues are fixed in the next phase, I will consider improving my score.
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 4: The contributions are significant, and do not exist in prior works.
empirical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
flag_for_ethics_review: NO.
recommendation: 5: marginally below the acceptance threshold

From: ICLR.cc/2023/Conference/Paper1631/Reviewer_dsY7
confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
summary_of_the_paper: The manuscript introduces a noisy interpretation of the standard LIF neuron, and shows that it can lead to probabilistic inference.  Notably, the surrogate gradient technique is shown to be a "special case" of this, meaning that SG can be interpreted probabilistically.
strength_and_weaknesses: Strengths
The main strength is the contribution that SG can be interpreted probabilistically.  That in turn justifies an otherwise ad-hoc hack.

Weaknesses
I find few.  If anything there is too much information in the the manuscript meaning that the figures are too compact.
clarity,_quality,_novelty_and_reproducibility: I think this is basically a good submission.  The main contribution is embedded in some otherwise difficult to follow prose and mathematics, but there is a core contribution that is novel and non-trivial.
summary_of_the_review: SG is the leading means of training spiking nets, but is quite ad-hoc.  The manuscript explains SG in terms of a probabilistic approach.  That is non-trivial and useful.
correctness: 4: All of the claims and statements are well-supported and correct.
technical_novelty_and_significance: 4: The contributions are significant, and do not exist in prior works.
empirical_novelty_and_significance: 4: The contributions are significant, and do not exist in prior works.
flag_for_ethics_review: NO.
recommendation: 8: accept, good paper