Paper: SOLO: Surrogate Online Learning at Once for Spiking Neural Networks (ICLR-2024)

Reviewer comment (Writing):
Lack of clarity: Some of the mathematical expressions lack proper definition and notation. I am confused by some details.

Author response:
We have endeavored to express the 'lack of proper definition and notation' issue within the equations as much as possible. Should there be any elements that were not adequately addressed, we have explained the relevant notations in Appendix A.1.~B.2..

Reviewer comment (Presentation):
Minor: the citation is not proper in the content. I think the author should use ‘\citep{}’ instead of ‘\cite{}’ most of the time.

Author response:
We implement the citation method you suggested. We are grateful for your guidance on this matter.

Reviewer comment (Theory):
For equation 6, why is there item $\theta(U^\sim[t]-\theta^\sim_{th})$ rather than $\theta(abs(U^\sim[t]-\theta^\sim_{th})<p)$.

Author response:
We have explored both scenarios and found that $\Theta(\tilde{U[t]}-\tilde{\theta_{th}})$ is more suitable. pPTRACE is ultimately proposed to output binary values named eligible spikes. As seen in Appendix A.4, pPTRACE is explained along with the clamp function, k, and the relevant information is present. The rationale behind using the clamp function is as follows:

 3-1. As forward propagation occurs, the generated spikes are added to the eligible potential. Unlike neurons, pPTRACE does not reset after a certain event (in the case of neurons, a spike), allowing it to accumulate increasing values. We need a limit range for binarization, hence the necessity for a clamp function (see Appendix Figure 1).

 3-2. While torch.clamp() could have been used to set a limit range, it is unable to create a gradient, thus making it ineffective in generating a gradient that flows with the time constant of pPTRACE.

Because pPTRACE doesn’t have any threshold and just accumulates potential, the Heaviside function is more suitable than a box range. Moreover, the Heaviside function is also suitable for hardware implementation as it can be implemented with just one comparator.

Reviewer comment (Theory):
Why does pTRACE need a clamp function $k$? I think equation (5) really ensembles the proposed pPLI (equation (2)). Why don’t you simply use pPLI as a surrogate?

Author response:
We have explored both scenarios and found that $\Theta(\tilde{U[t]}-\tilde{\theta_{th}})$ is more suitable. pPTRACE is ultimately proposed to output binary values named eligible spikes. As seen in Appendix A.4, pPTRACE is explained along with the clamp function, k, and the relevant information is present. The rationale behind using the clamp function is as follows:

 3-1. As forward propagation occurs, the generated spikes are added to the eligible potential. Unlike neurons, pPTRACE does not reset after a certain event (in the case of neurons, a spike), allowing it to accumulate increasing values. We need a limit range for binarization, hence the necessity for a clamp function (see Appendix Figure 1).

 3-2. While torch.clamp() could have been used to set a limit range, it is unable to create a gradient, thus making it ineffective in generating a gradient that flows with the time constant of pPTRACE.

Because pPTRACE doesn’t have any threshold and just accumulates potential, the Heaviside function is more suitable than a box range. Moreover, the Heaviside function is also suitable for hardware implementation as it can be implemented with just one comparator.

The pPLI neuron acts as a spike counter in SOLO, counting the network's output spikes, which is a completely different role from pPTRACE. Depending on the mode of leaky integration or integration, the pPLI neuron counts (or accumulates) spikes and is utilized during loss calculation.

Reviewer comment (Reproducibility):
How do we implement SOLO on a neuromorphic platform when it has a float-point derivation?

Author response:
The experimental data presented in the paper is implemented in a GPU-based floating-point computing environment. We have implemented the surrogate gradient using PyTorch's autograd and custom gradient functions.

Additionally, SOLO attempts to maintain the conventional SNN layout, which includes spike generators (or spike routers), weights, and a neuron core. Additional computational components such as pPLIF neurons, pPLI neurons (equivalent to spike counters), and surrogate strategies are designed for ease of hardware implementation. Therefore, if your platform has a conventional SNN layout and you have access to computing components for weight update, SOLO can function well on a floating-point-based neuromorphic platform, both automatically and manually. We are also very interested in digital neuromorphic platforms and are actively working to deploy SOLO on a board system. If you need any further information, we would be happy to assist.

Reviewer comment (Novelty):
The idea of the proposed method may, to some extent, be viewed as a special case of a recent work [3]. It proposes a method SLTT, which drops the temporal dependency of BPTT and only uses spatial gradients at each time step, and it further proposes a variant SLTT-k, which randomly samples k time steps for the spatial gradient. The method in this paper may be viewed as taking k=1 and fixing the considered time step as the last time step. However, this paper ignores gradients for previous time steps, leading to much poorer performance.

Author response:
We have reviewed the SLTT paper, and as you mentioned, its intent is somewhat similar to SOLO, and SOLO can be considered a special case of SLTT-K. However, SLTT-K performs backpropagation for randomly sampled K time steps at each training point, and there's no results of the case where K = 1 (minimum K = 2). Furthermore, SLTT-K ultimately requires memory to hold spatial states for specific K-time steps, making O(1) memory complexity impossible on neuromorphic platforms. In contrast, SOLO has the advantage of being able to directly access all components for weight updates at the final time step, transitioning to the weight update phase without the need for memory on neuromorphic platforms. In summary, in our work, SOLO demonstrates performance in harsher training configurations and hardware environments compared to SLTT.

Reviewer comment (Theory):
It is not clear enough why pPLIF is more straightforward for hardware implementation than PLIF. If consider deploying trained models, $\beta$ for the current in PLIF can be absorbed into the weight. If consider training models, it is also unclear for pPLIF how the gradient for the learnable membrane time constant can be calculated on hardware.

Author response:
You can find information on the PLIF neuron model in Appendix A.1, 'Neuron Model.' In the PLIF neuron model, the parameter \beta is shared for use both in the computation of membrane potential and in the modulation of current. \beta for the current couldn’t be ignored (or absorbed to current) when calculating the gradient with respect to \beta. In Appendix A.5, 'Related Gradient Chain,' you can find the process used to calculate the gradient with respect to the membrane constant in PLIF neurons, comparing it with pPLIF neurons and SOLO. On pPLIF neurons and SOLO. In the case of pPLIF neurons and SOLO, the gradient flowing through the time constant is simplified, which is expected to facilitate easier hardware implementation.

Reviewer comment (Evaluation):
In table 1 and table 2, the baseline performance looks not very high, e.g., for CIFAR 10, SNN SOTA is already close to 95% with 4-6 time steps, and CIFAR100 around 73%, but in these tables, these number are relatively low, could you explain the reasons?

Author response:
Furthermore, AutoSNN is originally composed of 13 convolutional layers and 1 fc layer, and through light-weighting, it is configured with 11 convolutional layers. (See Appendix Figure 4) However, recent state-of-the-art (SOTA) models employ bigger models such as ResNet 18, 34, 50, and others; hence, they are expected to exhibit higher accuracy on large datasets like CIFAR-100 and ImageNet.