From: ICLR.cc/2023/Conference/Program_Chairs
title: Paper Decision
decision: Reject
metareview:_summary,_strengths_and_weaknesses: Training spiking neural networks by backpropogation is nontrivial due to gradients being not well defined for spiking output.  A few approaches have previously been developed to solve this problem.  The authors offer a new solution strategy, adaptive surrogate gradients, which they present as an improvement over the existing surrogate gradients method.  

The initial reviews were rather negative, with 1/4 reviewers advocating for acceptance.  However, the scores did significantly improve during the discussion period and now tend weakly positive.  Reviewer AKTD found the approach interesting and clear enough, but not a substantial enough contribution and of only incremental novelty.  Reviewer tzdq found the approach simple and well-performing, but had other concerns, most of which the authors cleared up as misunderstandings.  Reviewer 5XQe saw the performance boost as too small in magnitude as well as had some issues with clarity and the theoretical analysis.  The authors addressed some comments, and the reviewer reconsidered whether the small performance boost may still be impactful.  Reviewer YpMg found the idea a novel approach for this setting, but also noted the only modest performance improvement.  They also noted some clarity and reference issues.  This reviewer also had an extensive back-and-forth with the authors clarifying the significance of the performance boost from the adaptive alpha parameter.  This was satisfactory to the reviewer and they maintained their score at 8.
justification_for_why_not_higher_score: This paper appears to be technically sound and potentially of legitimate interest to a small subset of the ICLR community.  However, given the limited magnitude of the improvement offered by this innovation (even the most enthusiastic reviewer sought substantial clarification due to the limited magnitude of the improvement), I think this is overall just below the threshold for acceptance.  
justification_for_why_not_lower_score: N/A
summary_of_AC-reviewer_meeting: I'll note that because the scores were still in flux around when scheduling of AC-reviewer discussions was taking place, so there was no video chat discussion for this one (i.e., it was initially below borderline and only later moved up to borderline).  This has been communicated to the SAC and PC.

From: ICLR.cc/2023/Conference/Paper4414/Authors
title: Thanks
comment: We are very glad that our responses can address your concerns. Thank you for your recognition of our work and your efforts on improving our work!

From: ICLR.cc/2023/Conference/Paper4414/Reviewer_YpMg
title: All clear now
comment: Thanks for the clarification.
I wasn't aware that you had done the condition "p=1, adaptive alpha", leading to 92.69 +-  0.04 (this did not appear in the table of your previous post, nor in the manuscript!)
Now I agree that the gain caused by p<1 is significant with adaptive alpha, even though it's modest.

Therefore I maintain my rating of 8.

From: ICLR.cc/2023/Conference/Paper4414/Authors
title: Further Explanation for Fig. 3b
comment: Thanks for your further question. Perhaps we did not clarify the results well.  To better understand the role of adaptive alpha and p<1 respectively, we add the detailed statistics under the case of **p=1, adaptive alpha** (Fig. 3b) and rearrange the table as follows:

|                   |       p=1        |      p=0.8       |
| :---------------: | :--------------: | :--------------: |
|  fixed $\alpha$   | 92.86 $\pm$ 0.05 | 93.03 $\pm$ 0.08 |
| adaptive $\alpha$ | 92.69 $\pm$ 0.04 | 93.01 $\pm$ 0.07 |

In the case of adaptive $\alpha$, the performance of $p=0.8$ exceeds the performance of $p = 1$ and their respective 95% confidence intervals are non-overlapping. It validates the effect of alternating forwarding, which enhances the capacity for generalization. The observation is also supported by the analysis in Section 4.3 which suggests that alternating forwarding could be considered as a regularization term. Interestingly,  as shown in the case of $p=1$, employing adaptive $\alpha$ directly in SG leads to the degradation of optimal performance instead even though it brings robustness on different $\alpha$ settings. It indicates the mismatching gradients in vanilla SG may not adjust the width properly when $\alpha$ has been selected properly.  In contrast, adaptive gradient learning and alternating feedforward in ASGL operate in a complementary manner and so have the advantage in terms of both network generalization and robustness for different $\alpha$ selections.

From: ICLR.cc/2023/Conference/Paper4414/Reviewer_YpMg
title: Still not clear whether p<1 helps
comment: Thanks for your answer.

If I understand correctly, Table 4 ASGL was done with p=0.8, and SG corresponds to fixed alpha and p=1.
Here it is clear that ASGL helps.
But we are not sure if it's due to adapting alpha or to p<1

Adapting alpha had been done before.
I would like to be sure that p<1, the main novelty of this paper, helps.
Unfortunately, Fig 3b and the table in your preceding post do not show a significant effect.
The 95% confidence interval of SG (mean +- 2*sem) overlaps with the ones of ASGL, so we cannot conclude.
Maybe you could do more seeds.
Alternatively, you could add another column to Table 4 corresponding to ASGL with p=1.

From: ICLR.cc/2023/Conference/Paper4414/Authors
title: Thank Reviewer AKTD and more supplements 
comment: Thank you so much for noticing our improvements. Your valuable comments help to improve our work. 

Due to the time limit of stage 1, we present preliminary results for the response. These days, we continue to increase the time step on both Tiny-ImageNet and CIFAR-10 datasets and make better comparisons with other methods. As shown in the modified Table 1 and Table 2, the proposed method achieves state-of-the-art across all the tested time steps. In addition, the performance tends to be saturated with the growth of the time steps.

Modified Table 1: Increasing time steps on Tiny-ImageNet dataset

|                    Method                    |      Type       | Architecture | Time steps | Top-1 Acc. |
| :------------------------------------------: | :-------------: | :----------: | :--------: | :--------: |
|      Spike-thrift (Kundu et al., 2021)       |     Hybrid      |    VGG-16    |    150     |   51.92    |
|           DCT (Garg et al., 2020)            |     Hybrid      |    VGG-13    |    125     |   56.90    |
| SNN Calibration (Li et al., 2021b) $\dagger$ |   Conversion    |    VGG-16    |     32     |   53.96    |
|       QCFS (Bu et al., 2021) $\dagger$       |   Conversion    |    VGG-16    |     32     |   53.54    |
|        Online LTL (Yang et al., 2022)        | Tandem Learning |    VGG-13    |     16     |   54.82    |
|       Offline LTL (Yang et al., 2022)        | Tandem Learning |    VGG-13    |     16     |   55.37    |
|                  This Work                   |      ASGL       |    VGG-13    |     8      |   56.81    |
|                  This Work                   |      ASGL       |    VGG-13    |     4      |   56.57    |

Table 2: Increasing time steps on CIFAR-10 dataset

|           Method/Time step           |        2         |        4         |        6         |
| :----------------------------------: | :--------------: | :--------------: | :--------------: |
|        STBP-tdBN (ResNet-19)         |      92.34       |      92.92       |      93.16       |
|           TET (ResNet-19)            | 94.16 $\pm$ 0.03 | 94.44 $\pm$ 0.08 | 94.50 $\pm$ 0.07 |
| SEW-ResNet (SEW-ResNet-18) $\dagger$ | 91.22 $\pm$ 0.14 | 94.39 $\pm$ 0.18 | 95.24 $\pm$ 0.11 |
|          Dspike (ResNet-18)          | 93.13 $\pm$ 0.07 | 93.66 $\pm$ 0.05 | 94.25 $\pm$ 0.07 |
|        This Work (ResNet-18)         | 95.27 $\pm$ 0.06 | 95.35 $\pm$ 0.25 | 95.55 $\pm$ 0.17 |

From: ICLR.cc/2023/Conference/Paper4414/Authors
title: Thank Reviewer tzdq and further answer
comment: We are very glad that our responses can address your concern and thank you sincerely for the reply and further suggestions. Following the advice, we will take more clarifications and explanations about the event-based inference in the revised version.  We would like to answer further questions in the following:
>  Given the stochastic learning, I believe that the accuracy deviation should be larger than other learning algorithms.  I wonder if the authors could explain these inconsistent trends.

Thanks for the thoughtful question. As shown in Table 1, there is indeed part of the experimental results which shows a smaller accuracy deviation compared to the other learning algorithm. It may be due to the proposed mechanism of adaptive width learning coupled with stochastic noise injection. In this way, although the accuracy deviation from stochastic learning is increased,  the accuracy deviation caused by manual width setting can be compensated to a certain extent. Therefore, the overall accuracy deviation shows randomness in the tradeoff between the two sources.

>  Also, the smaller the deviation, the larger the number of timesteps. For ResNet18 on CIFAR-10, the trend is opposite to the expectation while the others are in agreement with it. I wonder if the authors could explain these inconsistent trends.

Thanks for your insightful question. As far as we know, there is no theoretical analysis to explain the relationship between the deviation and time steps. Based on our experiments and recent research [1-3], it appears that the deviation will occasionally rise with the number of timesteps [1] while occasionally the contrary is true [2,3]. It may be related to both network architecture and training methods. The underlying causes require further research. 

[1] Deng, et al. "Temporal Efficient Training of Spiking Neural Network via Gradient Re-weighting." International Conference on Learning Representations. 2021.

[2] Xiao, et al. "Training feedback spiking neural networks by implicit differentiation on the equilibrium state." *Advances in Neural Information Processing Systems* 34 (2021): 14516-14528.

[3] Guo, et al. "Reducing Information Loss for Spiking Neural Networks." European Conference on Computer Vision. Springer, Cham, 2022.

>  the accuracy of the authors in Table 2 needs its deviation value as well.

Thanks for your suggestion. We have run the experiments under 4 different random seeds and supplemented the deviation value ($79.33 \pm 0.53$).

From: ICLR.cc/2023/Conference/Paper4414/Authors
title: Thank Reviewer  5XQe and further answer
comment: Thanks sincerely for your response and further questions. And we would like to address your concerns in the following:

> The second order approximation in the proof is not justified, this is problematic because this is exactly where training binary networks are difficult: the difference between the vector of probability and binary sample is not small !

We agree the derivation is just a second-order approximation that may lose information carried by high-order terms. However, this approximation provides insight into understanding the underlying reasons why noisy networks can converge toward SNNs. In this way, the layer-wise local activation distance could be extracted from global task-oriented loss. Some existing works [1-2] utilized similar approximation techniques for analyzing neural network learning algorithms. Furthermore, it is possible to explore high-order approximation within this framework. Additionally, the similarity analysis between noisy networks and SNNs also supports the approximation:  the difference $\Delta$ between the vector of probability and binary sample tends to decrease with the network training (Fig. 3e). Perhaps our previous statement has caused your misunderstanding, and we will revise the manuscript accordingly.

[1] Wei, et al. "The implicit and explicit regularization effects of dropout." *International conference on machine learning*. PMLR, 2020.

[2] Deng, et al. "Optimal conversion of conventional artificial neural networks to spiking neural networks." *International Conference on Learning Representations*. 2021.

>Please state somewhere that for T=1 the spiking network is equivalent to a binary feedforward network so this method also appears to work in the more generic context of binary neural networks and straight through estimators (spiking networks should not be a niche !)

Thanks for your suggestion! We will explain the relations between SNNs and binary networks and state that the method has the potential to be applied in the more generic context of binary neural networks and straight through estimators.

From: ICLR.cc/2023/Conference/Paper4414/Authors
title: Thank Reviewer YpMg and further answer
comment: Thanks sincerely for your response and further comments. Following your suggestion, we report the mean accuracy and s.e.m for each case as follows:

|                    | p=1, fixed $\alpha=1$ | p=0.8, fixed $\alpha$ = 1 | p=0.8, adaptive  $\alpha$ |
| :----------------: | :-------------------: | :-----------------------: | :-----------------------: |
| ResNet-18 (Fig 3b) |   92.86 $\pm$ 0.05    |     93.03 $\pm$ 0.08      |     93.01 $\pm$ 0.07      |
|     ResNet-19      |   93.70 $\pm$ 0.12    |     93.75 $\pm$ 0.20      |     93.96 $\pm$ 0.10      |

As illustrated in the table, the proposed method (p=0.8, adaptive  $\alpha$) still outperforms vanilla SG even for the best-selected alpha when $p=1$. Besides, another important contribution of this work is enabling adaptive width learning coupled with probabilistic forwarding and so exhibiting significant performance improvement for a large range of $\alpha$ (Table 4).

Table 4: Comparison on Image Classification

| Width ( $\alpha$ ) |  SG   | ASGL  |
| :----------------: | :---: | :---: |
|        0.5         | 93.19 | 94.11 |
|        1.0         | 93.78 | 94.30 |
|        2.5         | 90.68 | 94.09 |
|        5.0         | 62.34 | 93.61 |
|        10.0        | 30.85 | 93.53 |

Thanks for your positive feedback on our work. Your comments help us much when preparing the revised version of the manuscript. And please feel free to discuss if there is anything else to clarify.

From: ICLR.cc/2023/Conference/Paper4414/Reviewer_AKTD
title: Response to the rebuttal
comment: Thank you for providing more results. I checked the new results and the referred results. The current manuscript is improved. Based on these results, I would love to increase my rating to 6.

From: ICLR.cc/2023/Conference/Paper4414/Reviewer_tzdq
title: Thank the authors
comment: I just saw Line 7 in Algorithm 1. I recommend the authors to clarify the only event-based inference in the manuscript not just in Algorithm 1. 

$\textbf{Another question1:}$
Given the stochastic learning, I believe that the accuracy deviation should be larger than other learning algorithms. Also, the smaller the deviation, the larger the number of timesteps. For ResNet18 on CIFAR-10, the trend is opposite to the expectation while the others are in agreement with it. I wonder if the authors could explain these inconsistent trends.  

$\textbf{Another question2:}$ the accuracy of the authors in Table 2 needs its deviation value as well.

Overall, I satisfy with the authors answer so I will raise the rate slightly.

From: ICLR.cc/2023/Conference/Paper4414/Reviewer_5XQe
title: Thank you
comment: Thank you for addressing my comments.

On second thoughts I do increase my grade because I believe the marginal increase of performance may help the field of spiking network to move forward. Training a binary neural network is never easy anything trick can be welcome.

I am still not convinced that this paper deserves publication though. Here are my remaining problems:

- The second order approximation in the proof is not justified, this is problematic because this is exactly where training binary networks are difficult: the difference between the vector of probability and binary sample is not small !

- Please state somewhere that for T=1 the spiking network is equivalent to a binary feedforward network so this method also appears to work in the more generic context of binary neural networks and straight through estimators (spiking networks should not be a niche !)

From: ICLR.cc/2023/Conference/Paper4414/Reviewer_YpMg
title: I'm concerned about significance of the accuracy boost
comment: Thank you for your answers.

Figure 3 has been much improved, however, I'm now afraid that the accuracy boost of the proposed method is not significant.

Vanilla SG corresponds to p=1 and fixed alpha.
Could you please report (here) the mean accuracy and s.e.m. for this condition?

The proposed method works best for p=8, right?
Could you please report the mean accuracy and s.e.m. for this condition, with and without adaptive alpha?

That way we will appreciate if the boost is significant.
You could also perform some statistical tests.

From: ICLR.cc/2023/Conference/Paper4414/Authors
title:  Response to Reviewer YpMg
comment: > Fang et al 2021 is cited, but it should appear in Tables 1 and 2.

Thanks for pointing out it.  Results of (Fang et al 2021)  have been added to Table 1 and 2 in the revised version. 

[1] Fang, Wei, et al. "Incorporating learnable membrane time constant to enhance learning of spiking neural networks." *Proceedings of the IEEE/CVF International Conference on Computer Vision*. 2021.

> The references are incomplete.
>
> 1. Eventprop (https://www.nature.com/articles/s41598-021-91786-z), which computes the exact gradients (then no gradient mismatch!) should be discussed
> 2. SEW ResNet (https://arxiv.org/abs/2102.04159) should be included in the tables.

Thanks for pointing out the two exciting works. We conduct discussions in the Related Work and include the results of SEW ResNet in Table 1 and 2. Here, we use the publicly available code to produce the results on the CIFAR-10 dataset as the original paper does not provide that.  

> Will the authors share their code?

The initial version of the code has been released in the supplementary material. After adding enough comments, we will provide a cleaner version. 

> Eq 2: shows a bias term that is not described in the text.

We have added the corresponding description in the revised version.

> Eq 8: don't use the same variable name "x" inside and outside the integral

Thanks for pointing out this. We have corrected it.

> p4: "In this way, there is no mismatching problem, but at the same time lacks the information from spikes." Rephrase.
>
> p5: "the gradient mismatching will gradually diminish with adaptive alpha This is confusing. I think mismatch decreases if alpha->0, not if alpha is learnable.

We have revised the corresponding description to avoid ambiguity.

p4:  In this way, although there is no mismatching problem, it is difficult to guarantee the network dynamics evolving into that of SNNs finally.
p5: In this way, the gradient mismatching will gradually diminish when adaptive $\alpha$ approaches 0 against spike noise. 

> I suggest adding the number of trainable parameters of each model in Table 1, 2, 3 and 6.

Due to the time limit during the response, it is difficult to collate and calculate the number of trainable parameters used in all the referenced papers, especially for some customized network structures. We agree to include those results in the final version.

> Some typos: p7: "Perforamnce" p8: "Comparision" p9: "CIAFR"

Thanks for pointing out this. We have corrected them.

From: ICLR.cc/2023/Conference/Paper4414/Authors
title: Response to Reviewer YpMg
comment: Thank you for your positive and very detailed feedback. We are really encouraged that you find our method novel and achieve significant improvements. We would like to address your concerns and answer your questions in the following.

> Fig 3b is my favorite. We really see the advantage of alternating w.r.t. standard SG learning (p=1). Given that the accuracy boost is modest (~0.5%) I wonder if the authors could use error bars (with multiple seeds) to make sure that the boost is significant. Also, was this curve done with a trainable alpha? I would be curious to see if we still get a boost with a fixed alpha. Also, reporting the training accuracy would be useful (I suspect it decreases with p). Finally, the y-axis label is missing. 

Following your suggestion, we conduct experiments under three different seeds to ensure the performance boost is significant. As shown in revised Fig. 3b, the solid line and the shadow indicate the expectation and error, respectively.  We also report the results with fixed alpha in the same way. Here, the optimal hyper-parameter selection $\alpha=1$ (Table 4) is adopted to guarantee the effectiveness of alternating. Fig. 3b shows there is still a performance boost ($p=0.8$) even with a fixed alpha. The corresponding training accuracy with error bars is also reported in Fig. 3c. Just as you suspect, it decreases with $p$ for both cases. Moreover, we have added the missing y-axis label in this revision.

> It's unclear what value was used for p in sections 5.1, 5.2, and 5.3.

Most of the experimental results used a fixed p=0.8 including CIFAR-10, DVS-CIFAR10, and DVS128 Gesture while part of the results used the exponential scheduler to gradually increase $p$ during training such as MedlyDB and SHD. More details could be found in Appendix A.3 and Table 6 of the revised version.

> It's unclear how the reset is done. Eq 1 shows that the potential goes to zero (hard reset) if s=1. However, with an analog s, say s=0.2, then the new potential is the previous one multiplied by gamma*0.8? This should be discussed.

Yes, we use a 'soft-reset' mechanism for analog spikes $s$. To understand this intuitively, suppose the membrane potential $u[t]=0.2$, threshold $\theta=0.5$, and width $\alpha=1$. The analog spike $s[t]=\text{clip}(\alpha(u[t]-\theta)+0.5,0,1)=0.2$ represents the probability of the spike emission. And the different surrogate forwarding functions model the relation between expectational membrane potential and spike probability (rate) during a short period (corresponding to one time step).  From this perspective, the neuron has an 80% chance of not emitting a spike and maintaining the previous potential state. So $u[t]$ should be reset into  $0.8u[t]$ in the sense of expectation. Afterward, potential decay is done with $u[t+1]=\gamma u[t]$. We also add the relevant description in the main text.

From: ICLR.cc/2023/Conference/Paper4414/Authors
title: Response to Reviewer 5XQe
comment: > The theory is interesting and insightful but it lacks rigor. I believe that it should not be described as a "Theorem". A theorem traditionally invites normally for a rigorous mathematical proof and rather as a mathematical "analysis" which has to be taken with some grain of salt considering the approximations which are made in the appendix. These are my main issues with this proof:

We are very glad that you find the theoretical insight and thank you for pointing out the problem. We have modified the theorem into a proposition as well as the relevant descriptions for mathematical rigor. Here, we respond point by point to your concerns on theoretical analysis as follows:

> I think a factor 1/L is missing in equation (22) in Appendix

Thank you for your thoughtful comments. We have checked Eq. 22 (Eq. 23 in the revised version) carefully and do not find the possible entry introducing $1/L$. The expanded details are as follows:
$$
\begin{equation}
    \begin{aligned}
\ell_{\text{noise}}(F, s)&=\ell(F_\text{snn}(s))+ \mathbb E_{\hat{m}}[\ell (F_\text{snn}(s,\hat{m})) -\ell (F_\text{snn}(s))] \\\\
&=\ell (F_{\text{snn}}(s))+{ \mathbb E}_{\hat{m}}[\ell (G^0(s^0,\hat{m}))-\ell (G^L(s^L))] \\\\
&=\ell (F_\text{snn}(s)) +  \mathbb E_\hat{m} \left[ \sum\_{l=1}^L
\left({ \ell} ({G}^{l-1}({s}^{l-1},\hat{{m}}))-{\ell}({G}^l({s}^l, \hat{{m}}))\right)
 \right]  \\\\
&=\ell (F_\text{snn}(s)) + \sum\_{l=1}^L  \mathbb E_\hat{m} [{ \ell} ({G}^{l-1}({s}^{l-1},\hat{{m}}))-{\ell}({G}^l({s}^l, \hat{{m}}))]\\\\
&= \ell (F_\text{snn}(s)) + \sum\_{l=1}^L R(G^l,s^l)
    \end{aligned}
  \end{equation}
$$
Intuitively, the divergence between the hybrid network and SNN can be decomposed into the sum of layer-wise activation distance by constructing ${G}^l$.

> The Taylor expansion is performed although the Delta has no reason to be small apriori as far as I can see. 

We agree that the Taylor expansion is performed in an approximate form and the higher-order terms are discarded for simplification. The corresponding description has been modified for mathematical rigor. Meanwhile, the range of  $|\Delta|=({1}-{\hat{m}}^l) \odot (H_\alpha(\hat{{u}}^l)- \Theta(\hat{{u}}^l) )\in [0,\text{max}(\frac{1-p}{2p},\frac{1}{2})]$ is also added. 

> The looks more like an approximate mathematical analysis which is perhaps insightful but the equality prominently described as a theorem in the main text is not a true equality. In my opinion, it is not ok to call that a theorem although it is insightful like so it should be presented differently in the main text.

Thanks for your suggestion. We have revised the description and presented the mathematical analysis rigorously.

From: ICLR.cc/2023/Conference/Paper4414/Authors
title: Response to Reviewer 5XQe
comment: Thanks for your thoughtful and constructive comments. And we would like to address your concerns in the following:

> 1. In table 4 and table 5, I believe that we see most accurately the benefits of the training method. The fact that ASGL is robust with high width is rather trivial, since the alpha is further optimize in this case. But I find more informative that the ASGL performs 1.5% better than SG with the optimal width (alpha=1) in Table 4 and identically for image reconstruction in Table 5. This indicates that ASGL gives a meager improvement over a tuned SG which is not bad but not extremely impressive considering the additional hyperparameters that are being used.


Although the performance improvement is not extremely significant, we believe the most significant insight this work provides is a new perspective to train SNNs directly by combining surrogate forwarding and probabilistic spike injection. It has a compelling feature as it solves the problems of gradient mismatching in surrogate gradient and shows extraordinary robustness for different hyperparameter selections of smooth factors. Besides this, even though ASGL introduces a new hyperparameter $p$, it incorporates smooth degree $\alpha$  (hyperparameter in surrogate gradient) into the training of a hybrid network at the same time. In general, the number of hyperparameters does not increase. Meanwhile, comparing Fig 3b and Table 4,  the hyperparameter $p$ in ASGL is more robust compared to the width $\alpha$ in surrogate gradient.  

> The Figure 3 seems interesting to explain what is happening during training. I find the Figure 3 a) very surprising and interesting and there is perhaps something deep to understand behind that. I read the hypothetical explanation of the author for this, interesting point of view, maybe it's true. I find the other panels either trivial or hard to understand because there is no caption to explain what it means (I tried but I could not guess what is saying panel (e)).

We are really encouraged that you find Figure 3a surprising and interesting. To better explain other panels of Figure 3, we have added the corresponding caption to each subfigure. For panel (e), we want to illustrate the noisy network will evolve towards SNN even for fixed noisy probability $p$. After each training epoch with ASGL, we record layer-wise activations of the noisy network and the embedded SNN with shared weights for each sample, and calculate the average cosine similarities $S$ over all layers. Panel (e) reports the mean and standard deviation of $S$ across all samples in the training set of CIFAR-10. Both $p=0.8$ and $p=0.5$ are reported. The results show the training with ASGL makes the hybrid network move towards SNN and so the similarities grow up to around 0.8 while overall similarities are relatively low at the beginning even with the shared weights.  The decremental standard deviation also supports this argument.

From: ICLR.cc/2023/Conference/Paper4414/Authors
title: Response to Reviewer tzdq
comment: Thanks for your efforts and time to help improve the quality of this work. There are also some misunderstandings that may affect your evaluation of this work. And we would like to address your concerns in the following:

> The neural net for ASGL is not a spiking neural net given that the operation is not based on asynchronous events but based on real-valued outputs calculated at every timestep. Therefore, the performance comparison of ASGL with other SNN learning algorithms based on 1b spike event packets is NOT fair at all. The neural net for ASGL is rather close to DNN given that real-valued outputs forward propagate through all layers at every timestep. Further, this model is not suitable for neuromorphic hardware based on asynchronous binary event propagation. Unlike the authors’ claim that ASGL simultaneously addresses the advantages of whetstone and SNN, the model and methods rather address their disadvantages.

Firstly, our method is **evaluated based on 1b spike event packets without real-valued activations**, thus our comparison with other SNN algorithms is fair. As explained in the original manuscript (lines 4-6 on page 5),  full spikes are used during validation while real-value activations are only added with a stochastic mask during training with ASGL, which could be verified further in Algorithm 1. Therefore, the network after training could be applied to neuromorphic hardware and obtain the advantages from asynchronous binary event propagation. Secondly, using real-valued information is a widely adopted solution for training deep SNNs [1-4] which usually requires information from real-value gradients. This work invented an alternative way to use analog activation information to train deep SNNs, which has the advantage compared to the surrogate gradient learning in terms of both network generalization and robustness for different hyperparameter selections of smooth factors.

[1] Shrestha, et al. "Slayer: Spike layer error reassignment in time." *Advances in neural information processing systems* 31 (2018).

[2] Wu, et al. "Direct training for spiking neural networks: Faster, larger, better." *Proceedings of the AAAI Conference on Artificial Intelligence*. Vol. 33. No. 01. 2019.

[3] Neftci, et al. "Surrogate gradient learning in spiking neural networks: Bringing the power of gradient-based optimization to spiking neural networks." IEEE Signal Processing Magazine 36.6 (2019): 51-63.

[4] Fang, et al. "Incorporating learnable membrane time constant to enhance learning of spiking neural networks." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021.

> For a fair comparison, ASGL should be compared with whetstone and other learning algorithms for DNNs rather than SNNs. If exist, the advantages of ASGL over DNNs should be addressed, e.g., computational/space complexities, accuracy gain, and so forth

As explained above, our solution ultimately yields a network that is **entirely spike-based** without sacrificing asynchronous event-based calculation. It is suitable for neuromorphic hardware implementations and achieves asynchronous event propagation with binary activation.  Besides, the proposed method could incorporate the smoothness factor into the training without the need for a specific design of hyperparameter search, which could be computationally expensive in surrogate gradient. Moreover, the generalization is improved remarkably across all the tested benchmarks as shown in Table 1-5.

From: ICLR.cc/2023/Conference/Paper4414/Authors
title: Response to Reviewer AKTD
comment: > Since the preactivation values of two networks are the same membrane potentials, their output cosine similarity will be very high. Why not directly illustrate the results of the latter loss term of Eq 13?

 Following your suggestion, we approximate the latter loss by calculating $\ell_{\text {noise }}(F, s)-\ell_{\text{snn}}(F, s)$ here. As $\ell_{\text{noise}}(F, s) \triangleq \mathbb E_{\hat{m}}\left[\ell\left(F_{\text{noise}}(s)\right)\right]
$ is the expectation on mask matrix $\hat{{m}}$, we perform 20-time inference with the noisy network $F_\text{noise}$ and record the average loss as an approximation for $\ell_{\text {noise }}({F}, {s})$. The reason we do not calculate the latter loss directly is: (1) Calculating $D^{2}[{s}^l]$  (the second derivative of constructed network ${G}^l$ with respect to $s^l$ is computationally heavy.  (2)  We need to sample enough masks to estimate $\mathbb E_{\hat{m}}[ {G}^l]$. As a result, the $l2$-norm of estimated loss declines gradually from 1.42e-2  to  3.20e-3 during 300 epoch training. It verifies the penalty effect of the last term and shows $\ell_{\text {noise }}({F}, {s}) \to \ell_{s n n}({F}, {s})$ through training such a hybrid network.

Moreover, we would like to clarify that the pre-activation potentials of the two networks are different except that of the first layer and the first time step. During training, deviation error on the spike output is introduced by randomly substituting $\Theta(x)$ with $H_\alpha(x)$. The error will accumulate in both temporal and spatial domains and make the subsequent pre-activation potentials different. Therefore, as seen in Fig. 3e, the output cosine similarity is low  (~0.24 ) before training though the weights are shared between two networks.  After training, the cosine similarity exceeds 0.8 which also verifies the hybrid network approaches SNN even under fixed noisy probability $p$.

> Is there any use of recurrent connections in the experiments in this paper? Apart from appendix A.5, I do not see the recurrent connections. 

We use recurrent connections on the SHD experiment for a fair comparison. It has been illustrated in section 5.2 of the original version (**Performance on Sound Datasets**).

From: ICLR.cc/2023/Conference/Paper4414/Authors
title: Response to Reviewer AKTD
comment: Thanks for the comments and suggestions for improvement sincerely.  Please see our response below and the revised version of the manuscript. 

> The symbols in Section 4.3 are not very clearly explained.

We have revised Section 4.3 and carefully checked each symbol in Section 4.3 to ensure that the corresponding definition or description is available. In particular, we expand on the description of ${G}^l$ in the revised version. Define the $l$-th noisy spiking layer with random mask $\hat{{m}}^l$ as the noisy operator ${g}^l$. ${G}^l$ denotes the composite of the last $L-l$ noisy layers ${g}^l$ where $L$ is  overall layer number. For ease of reading, we also include the symbol table as shown in Table 13.


 Table 13.  The symbols and corresponding definitions (explanations). 
|           Symbol           |                         Definition                          |
| :------------------------: | :---------------------------------------------------------: |
|           $\ell$           |                        loss function                        |
|        $s$ or $s^0$        |                     input spike pattern                     |
|           $s^l$            |               the output spikes at l-th layer               |
|           $f^l$            |      the $l$-th spiking layer with $\Theta(\hat{u}^l)$      |
|           $g^l$            | the $l$-th noisy spiking layer with random mask $\hat{m}^l$ |
|           $G^l$            |       $g^{L} \circ g^{L-1}  \circ ... \circ g^{l+1}$        |
|            $p$             |   noisy probability controlling the percent of spike mode   |
| $\hat{{m}}$ | mask matrix normalized by $1/p$ |

> This paper only experiments on the very small time steps (e.g.1、2) and lack of some experiments on slightly larger time steps (e.g. 4、6) to make better comparisons with other methods. 

Following your suggestion, we have supplemented the experimental results for 4 time steps as listed in Table 1.  In general, there are still remarkable performance improvements compared with state-of-the-art methods. Additionally, it tends to become saturated with the increase of time steps suggesting the representation of more time steps can reduce the smoothness error. It is worth noting that we also evaluate the proposed method on the long time-step benchmark like SHD (250 time steps), MedlyDB (500 time steps) and verify its effectiveness. 

> Lack of experimental results on ImageNet to verify the method. 

Due to time constraints, we cannot provide experimental results on the ImageNet dataset. As an alternative, we test our method using the challenging Tiny-ImageNet dataset of  200 categories, which consists of 100,000 training images and  10,000 validation images with a 64x64 pixel resolution. The proposed method still achieves competitive experimental results with fewer time steps, as demonstrated in Table A.8, further validating the approach. In addition to the performance improvement, we believe our approach brings a dual perspective for SNN training compared to surrogate gradient that is surrogate forwarding with the injection of spike noise. In this way, the learning of smoothness can be realized naturally.


Table A.8  Comparison on Tiny-ImageNet dataset. 

|                    Method                    |      Type       | Architecture | Time steps | Top-1 Acc. |
| :------------------------------------------: | :-------------: | :----------: | :--------: | :--------: |
|      Spike-thrift (Kundu et al., 2021)       |     Hybrid      |    VGG-16    |    150     |   51.92    |
|           DCT (Garg et al., 2020)            |     Hybrid      |    VGG-13    |    125     |   56.90    |
| SNN Calibration (Li et al., 2021b) $\dagger$ |   Conversion    |    VGG-16    |     32     |   53.96    |
|       QCFS (Bu et al., 2021) $\dagger$       |   Conversion    |    VGG-16    |     32     |   53.54    |
|        Online LTL (Yang et al., 2022)        | Tandem Learning |    VGG-13    |     16     |   54.82    |
|       Offline LTL (Yang et al., 2022)        | Tandem Learning |    VGG-13    |     16     |   55.37    |
|                     Ours                     |      ASGL       |    VGG-13    |     4      |   56.57    |

From: ICLR.cc/2023/Conference/Paper4414/Authors
title: General Comment and Updated Manuscript 
comment: We thank all reviewers for their constructive reviews and suggestions.  We have put in a major effort to address all their comments. Here, we summarize our main revision of the manuscript as follows: 

* According to the reviewer’s suggestions, we supplemented the results on both CIFAR datasets at 4 time steps and more comparisons on the Tiny-ImageNet dataset.  
* The theoretical analysis in sections 4.3 and A.9 was stated more clearly and rigorously in the text. 
* We expanded more explanations and analysis in Fig. 3  with multiple seeds. 
* We included full discussion of the reset mechanism under noisy forwarding. 
* Phrasing clarifications and more comparisons with suggested literature are provided in the main text.

Below please find the responses to some specific comments.

From: ICLR.cc/2023/Conference/Paper4414/Reviewer_AKTD
confidence: 5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.
summary_of_the_paper: This paper analyzes the gradient mismatching problem in direct training neural networks and proposes a novel method to solve this problem by fusing the learnable relaxation degree into the network with random spike noise and get good results on both static and events datasets.
strength_and_weaknesses: Strengths:
1.	It is a novel and simple approach in the field of training spiking neural networks.
2.	The presentation of this paper is generally good and easy to read.
3.	The method proposed in this paper has a good performance.

Weaknesses:
1.	1. The symbols in Section 4.3 are not very clearly explained.
2.	This paper only experiments on the very small time steps (e.g.1、2) and lack of some experiments on slightly larger time steps (e.g. 4、6) to make better comparisons with other methods. I think it is necessary to analyze the impact of the time step on the method proposed in this paper.
3.	Lack of experimental results on ImageNet to verify the method.

Questions:
1.	Fig. 3 e. Since the preactivation values of two networks are the same membrane potentials, their output cosine similarity will be very high. Why not directly illustrate the results of the latter loss term of Eqn 13?
2.     Is there any use of recurrent connections in the experiments in this paper? Apart from appendix A.5, I do not see the recurrent connections.

clarity,_quality,_novelty_and_reproducibility: Overall this paper is well written and easy to follow. The originality exists but is not sufficient enough given that the problem itself was identified in another paper and not extended into a more general style in the current paper. The view could be broadened. 
summary_of_the_review: Overall, this is an interesting paper that adds a new paragraph to improve the training of spiking neural network. The originality exists but is not sufficient enough given that the problem itself was identified in another paper and not extended into a more general style in the current paper. Also, the performance examination needs to be improved to support its validity on large dataset and compatibility with other existing pipelines like the augmentation. 
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
empirical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
flag_for_ethics_review: NO.
recommendation: 6: marginally above the acceptance threshold

From: ICLR.cc/2023/Conference/Paper4414/Reviewer_tzdq
confidence: 5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.
summary_of_the_paper: The authors propose a method to modify Whetstone addressing the forward-backward mismatch issue (arising from the use of surrogate gradient functions) such that the information conveyed by spikes probabilistically contributes to the forward propagation of information. This method is referred to as adaptive smoothing gradient learning (ASGL). ASGL is implemented by a simple modification of the previous computational graph so that the additional computational cost is marginal. The performance of ASGL outperforms the previous algorithms on various datasets, CIFAR-10/100, DVS-CIFAR10, SHD.
strength_and_weaknesses: Strength:

The proposed method is simple and in need of marginally additional computational cost. ASGL achieves the SOTA accuracy on static datasets (CIFAR-10/100) and event datasets (DVS-CIFAR10 and SHD), which is somewhat arguable though.

Weaknesses:

1. The neural net for ASGL is not a spiking neural net given that the operation is not based on asynchronous events but based on real-valued outputs calculated at every timestep. Therefore, the performance comparison of ASGL with other SNN learning algorithms based on 1b spike event packets is NOT fair at all. The neural net for ASGL is rather close to DNN given that real-valued outputs forward propagate through all layers at every timestep. Further, this model is not suitable for neuromorphic hardware based on asynchronous binary event propagation. Unlike the authors’ claim that ASGL simultaneously addresses the advantages of whetstone and SNN, the model and methods rather address their disadvantages. 

2. For a fair comparison, ASGL should be compared with whetstone and other learning algorithms for DNNs rather than SNNs. If exist, the advantages of ASGL over DNNs should be addressed, e.g., computational/space complexities, accuracy gain, and so forth.

clarity,_quality,_novelty_and_reproducibility: Clarity: the paper is well organized, and the presentation is clear.

Reproducibility: the authors' providing the code is appreciated.

Novelty: ASGL is a modification of whetstone regarding the probabilistic address of spike information. The idea itself is novel.

Quality: Given the issues addressed in the above section, the significance of the proposed work is marginal. 
summary_of_the_review: The neural net considered in this work is not a spiking neural net, but it probabilistically depends on spike information as well. The activation is not in a form of binary spike, instead real-valued data. Additionally, the activation from each node is evaluated and conveyed to the postsynaptic neurons at every timestep, so that it should not be referred to as event. Therefore, the comparison of ASGL with other SNN algorithms is not fair at all. Particularly, the model is not suitable for any neuromorphic hardware, which considerably undermines the significance of the present work. 
correctness: 2: Several of the paper’s claims are incorrect or not well-supported.
technical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
recommendation: 6: marginally above the acceptance threshold

From: ICLR.cc/2023/Conference/Paper4414/Reviewer_5XQe
confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
summary_of_the_paper: The authors describe a training method for SNNs where the binary activity is mixed with the smoothed sigmoid-like activation with a stochastic mask. The smoothed activation corresponds to the true network for which the surrogate gradient of the SNN is exact. Throughout training, the mask is released to go from a purely smoothed function at the first epochs towards a full SNN at the end. Some theoretical analysis justifies this training approach and the performance is studied empirically on many tasks.
strength_and_weaknesses: The performance seems to improve with the suggested method but as far as I can see it is not ground breaking:

1) In table 4 and table 5, I believe that we see most accurately the benefits of the training method. The fact that ASGL is robust with high width is rather trivial, since the alpha is further optimize in this case. But I find more informative that the ASGL performs 1.5% better than SG with the optimal width (alpha=1) in Table 4 and identically for image reconstruction in Table 5. This indicates that ASGL gives a meager improvement over a tuned SG which is not bad but not extremely impressive considering the additional hyperparameters that are being used.

In the other method, the method is compared with other models and seem to out perform all of them, but I understand that this probably because the authors used very strong baselines with other tricks which are independent of the ASGL method described prominently in the publication.

2) The Figure 3 seems interesting to explain what is happening during training. I find the Figure 3 a) very surprising and interesting and there is perhaps something deep to understand behind that. I read the hypothetical explanation of the author for this, interesting point of view, maybe it's true. I find the other panels either trivial or hard to understand because there is no caption to explain what it means (I tried but I could not guess what is saying panel (e)).

The theory is interesting and insightful but it lacks rigor. I believe that it should not be described as a "Theorem". A theorem traditionally invites normally for a rigorous mathematical proof and rather as a mathematical "analysis" which has to be taken with some grain of salt considering the approximations which are made in the appendix. These are my main issues with this proof:

4) I think a factor 1/L is missing in equation (22) in Appendix

5) The Taylor expansion is performed although the Delta has no reason to be small apriori as far as I can see.

6) The looks more like an approximate mathematical analysis which is perhaps insightful but the equality prominently described as a theorem in the main text is not a true equality. In my opinion, it is not ok to call that a theorem although it is insightful like so it should be presented differently in the main text.

clarity,_quality,_novelty_and_reproducibility: The paper is clear and understandable and reasonably well written. The results are certainly reproducible given that it does not seem to vary so much from the SG baseline. 
summary_of_the_review: Improving surrogate gradient training is very important both for the SNN community and on the broader topic of quantized networks. The hypothesis is reasonable, but the performance does no look very good when looking carefully (see my comment above). 

Given that there is no reason for this method to be specific to SNN and not to apply on binary networks, I think a good paper on the topic should also cover the background literature on straight through estimators and quantized networks which gave birth to surrogate gradient methods and have now gone deeper than SNN research in the theoretical side.
correctness: 2: Several of the paper’s claims are incorrect or not well-supported.
technical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
recommendation: 5: marginally below the acceptance threshold

From: ICLR.cc/2023/Conference/Paper4414/Reviewer_YpMg
confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
summary_of_the_paper: The authors propose a new method to train spiking neural networks (SNNs) directly. The most efficient current method to do so is surrogate gradient (SG) learning where for the spike threshold the Heaviside step function is used in the forward pass, but a surrogate gradient (usually the derivative of a sigmoid function) is used in the backward pass instead of the true gradient, which causes "gradient mismatch". Here the authors propose that in the forward pass, one can alternate randomly between the Heaviside function and the antiderivative of the SG during training, while inference is always done with the Heaviside (otherwise it wouldn't be a SNN!). In terms of pros and cons:

Heaviside:
Pros: it is the function that will be used during the inference
Cons: gradient mismatch

Antiderivative of the SG:
Pros: no gradient mismatch
Cons: analog activations instead of spikes, which creates a discrepancy with inference.

By alternating between the two, the authors hope to take the best of both worlds, and they demonstrate that the idea seems to work in many experiments. 

The authors also learn the width of the SG (alpha), but this has been done before.
strength_and_weaknesses: STRENGTHS:

As far as I know, the alternating idea is new, and it seems to improve accuracy significantly at a marginal cost.

WEAKNESSES:

Fig 3b is my favorite. We really see the advantage of alternating w.r.t. standard SG learning (p=1). Given that the accuracy boost is modest (~0.5%) I wonder if the authors could use error bars (with multiple seeds) to make sure that the boost is significant. Also, was this curve done with a trainable alpha? I would be curious to see if we still get a boost with a fixed alpha. Also, reporting the training accuracy would be useful (I suspect it decreases with p). Finally, the y-axis label is missing.

It's unclear what value was used for p in sections 5.1, 5.2, and 5.3.

It's unclear how the reset is done. Eq 1 shows that the potential goes to zero (hard reset) if s=1. However, with an analog s, say s=0.2, then the new potential is the previous one multiplied by gamma*0.8? This should be discussed.

Fang et al 2021 is cited, but it should appear in Tables 1 and 2.

The references are incomplete.
* Eventprop (https://www.nature.com/articles/s41598-021-91786-z), which computes the exact gradients (then no gradient mismatch!) should be discussed
* SEW ResNet (https://arxiv.org/abs/2102.04159) should be included in the tables.

clarity,_quality,_novelty_and_reproducibility: Will the authors share their code?

Eq 2: shows a bias term that is not described in the text.

Eq 8: don't use the same variable name "x" inside and outside the integral

p4: "In this way, there is no mismatching problem, but at the same time lacks the information from spikes." Rephrase.

p5: "the gradient mismatching will gradually diminish with adaptive alpha"
This is confusing. I think mismatch decreases if alpha->0, not if alpha is learnable.

I suggest adding the number of trainable parameters of each model in Table 1, 2, 3 and 6.

Some typos:
p7: "Perforamnce"
p8: "Comparision"
p9: "CIAFR"

summary_of_the_review: A new and promising method for direct training of spiking neural networks.

correctness: 4: All of the claims and statements are well-supported and correct.
technical_novelty_and_significance: 4: The contributions are significant, and do not exist in prior works.
empirical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
flag_for_ethics_review: NO.
recommendation: 8: accept, good paper