Paper: Composing Recurrent Spiking Neural Networks using Locally-Recurrent Motifs and Risk-Mitigating Architectural Optimization (ICLR-2024)

Reviewer comment (Novelty):
The major contributions of this paper seem to be just taking the current DARTS-based architecture search method used in DNNs and using it for RSNNs. I would recommend the authors to highlight their key contributions (see Questions for details).

Author response:
We acknowledge that our current approach is based on DARTS. Our novelty lies in: we design the SC-ML layer to reduce the size of the network search space while ensuring the sparsity of recurrent connections. At the same time, in order to optimize this topology, we designed the HRMAS algorithm including IP to stabilize the network
function and mitigate potential risks caused by architectural changes.

Reviewer comment (Experiments):
Since the authors use the same topology for all the motifs, it would be good if the authors could elucidate how they chose this topology and what effect it has on the final architecture. I feel this notion of motifs is inspired by the blocks building the cells in the DARTS paper - if so, the DARTS method used a variety of different convolution layers for these blocks. It would be interesting if the authors could highlight why no such heterogeneity would be required in designing RSNNs.

Author response:
We admit that the concept of motif is similar to the cell in DARTs as the most basic computing unit. It is worth noting that both our SC-ML and HRMAS methods can be seamlessly extended to RSNN with different motif sizes. In the paper, we mainly consider computational complexity: heterogeneous motifs will bring significantly larger search spaces and reduce the efficiency of the NAS algorithm. By limiting motifs to the same size, we achieve a balance between search space size, network topology sparsity, and performance.

Reviewer comment (Novelty):
The SC-ML architecture seems very similar to the concept of clustered ESNs. Can you give some explanation of how this is similar/different?

Author response:
Our network does share topological similarities with ESN: there are recurrent connections within a single neuron layer, which complicates the spatiotemporal dynamic nature of the neuron layer and increases its information processing capabilities. However, the weights of most ESN networks are randomly set and not optimized, which significantly limits the learning capabilities of these networks. Our method can make all weight parameters in the network learnable by designing the SC-ML topology and HRMAS optimization method. This significantly improves the perfermence and applicability of RSNN.

Reviewer comment (Theory):
The authors introduce the HRMAS as a bi-level optimization where, in the first step, they optimize $\alpha$ and $w$ hierarchically, based on gradient-based optimization. In the second step, they use IP to adapt the parameters of each neuron over a time window. However, that does not match with the problem formulation in Eqs 1-3, where it seems the first-level optimization searches for the architecture, the second-level searches for the optimal parameters, and the third level optimizes the weights.

Author response:
We would like to point out that eq1-3 are only the complete theoretical form of the entire optimization problem, instead of the specific optimization process. Our bi-level optimization method is an approximate solution of this problem to alleviate the computational overhead.

Reviewer comment (Reproducibility):
From my personal experiences, such architecture search models are extremely unstable and highly dependent on the initializations. Can you add some details on the initialization you used for your experiments and whether two different initialization models converge to similar or very different models? It is also important that the authors rerun the experiments a few times (with same/different initializations) and report the mean and variance of the performance in Table 1.

Author response:
We rerun the experiment 5 times using different random seeds for initialization. The results are as follows. It can be seen that our optimization algorithm is not sensitive to the choice of initialization and can converge stably. We also provided more data in general response, more figures of complete learning curve with different initializations in Section: Additional Information in Appendix of our paper. 

| Dataset     | Learning Rule | Hidden Layers | Mean      | Std      | Best        |
|-------------|---------------|---------------|-----------|----------|-------------|
| TI46-Alpha  | TSSL-BP       | 800           | 96.08%    | 0.27%    | **96.44%**  |
| N-TIDIGITS  | TSSL-BP       | 400           | 94.27%    | 0.35%    | **94.66%**  |
| DVS-Gesture | TSSL-BP       | P4-512        | 88.40%    | 1.71%    | **90.28%**  |
| N-MNIST     | TSSL-BP       | 512           | 98.60%    | 0.08%    | **98.72%**  |

Reviewer comment (Presentation):
The figure 4 is not clear to show the mechanism of the HRMAS. More explanation can be added. Also the figure 2 is not clear enough to show the architecture optimization in HRMAS.

Author response:
Figure 2 shows the architecture search process in HRMAS. We first determine the motif size in the supernet, and then determine the connection types of intra and inter-connection (excitatory, inhibitory and non-existent). Figure 4 shows the complete optimization process of HRMAS: horizontally, we now use BP and IP to alternately optimize the weight parameter $\omega_1$ and neuron hyperparameter $\beta$ under a certain architecture $\alpha$, and obtain the trained weight parameter $\omega_1'$, $\omega_1''$ in this process. After this process is completed, we use BP to update the network architecture parameter $\alpha$ to obtain $\alpha'$ and repeat the above optimization process under the new architecture parameter $\alpha'$.

Reviewer comment (Evaluation):
Since the architecture optimization proposed in this paper is complex, how about the training speed and computation resources consumption? Only accuracy comparison seems not sufficient for ICLR publication.

Author response:
The proposed HRMAS bi-level optimization process is similar to DARTS, so the overall computational complexity is similar to DARTS; the IP method is an localized unsupervised learning method and does not constitute significant computational consumption. Furthermore, our proposed SC-ML topology greatly reduces the search space. Specifically, as the Figure9 in Appendix  shows, the HRMAS optimization of a SC-ML layer, with $n$ neurons, a motif size of $s$ and $n/s$ motifs, reduces the parameters that need to be optimized for the recurrent connection matrix from $O(n^2)$ to $O(sn)$: $O(n/s)$ inter-motif connections + $O(n/s * s^2)$ intra-motif connections + $O(n)$ neuron hyperparameters. Generally, $s \ll n$, which reduces the parameter space of recurrent connections to linear growth with the neuron numbers, allowing our algorithm scale well. Specifically, a complete training process of a RSNN with 800 neurons hidden layer for TI46-alpha dataset, including 150 epoch for cell size search, 150 epoch for connection type search and 400 epoch for finetune, takes 4 hours on single NVIDIA GeForce RTX 3090 GPU.

Reviewer comment (Experiments):
If the proposed RSNN model is applied to larger datasets such as DVS-CIFAR10, how about the training speed and computation resources consumption?

Author response:
Our proposed method is highly scalable. As the Figure 9 in Appendix shows, for a recurrent connection layer with n neurons, our method essentially reduces the optimization problem of a fully connected matrix with $O(n^2)$ parameters to $O(n)$ level. This keeps the cost of network architecture search within a controllable range when the complexity of the data set and the size of the network increase.

Thanks for your insightful question! We'd be glad to discuss any unclear details further.

Reviewer comment (Experiments):
The choice of baselines across different datasets lacks uniformity. The rationale behind using different baselines for each dataset remains unclear.

Author response:
The criterion for our selection of baselines is to align the recurrent spiking neural networks architecture. We acknowledge that the comparison with unsupervised algorithms (e.g., STDP in HeNHeS) (Chakraborty \& Mukhopadhyay, 2023) lacks fairness, but it is not the primary comparison. Except for this work, all other baselines use supervised algorithms for learning.

Reviewer comment (Experiments):
Can the authors provide insights into the evolution of motif topologies during training? Such an evolution could offer valuable insights into the algorithm's adaptability and optimization process.

Author response:
Here we provide a mathematical analysis of the recurrent connection matrix of SC-ML to provide more insights into motif topologies. For the fully connected recurrent weight matrix, the number of parameters is large and difficult to optimize. In order to solve this problem, we design SC-ML's topologies to turn recurrent matrix into a Band Matrix and use HRMAS to optimize it. Specifically, intra-motif connections constitute the sub-matrices on the diagonal of the recurrent matrix; inter-motif connections constitute the values on several sub-diagonals of the recurrent matrix; motif-size determines the size of the diagonal submatrix. In the network architecture search stage: in the first step we determine the size of its diagonal submatrix (motif size), and in the second step we determine the connection type on the band (inter and intra-motif topologies), including excitability and inhibition and non-existent connections. We then used BP to optimize the specific values of the excitatory connections in the band, and used IP to optimize neuron hyperparameters to slow down performance changes caused by network architecture changes.

The aforementioned configuration was designed to allow flexibility in constructing a high-performance sparse spiking RNN, while simultaneously constraining the search space to ensure practical viability in solving the complex optimization problem associated with RNN architecture. The proposed optimization framework is highly flexible and can accommodate other types of sparsity constraints.

Reviewer comment (Experiments):
There is inconsistency in the choice of baselines across datasets. The comparison of unsupervised learning rules with supervised ones seems inappropriate. As anticipated, unsupervised methods would underperform compared to their supervised counterparts.

Author response:
The criterion for our selection of baselines is to align the recurrent spiking neural networks architecture. We acknowledge that the comparison with unsupervised algorithms (e.g., STDP in HeNHeS) (Chakraborty \& Mukhopadhyay, 2023) lacks fairness, but it is not the primary comparison. Except for this work, all other baselines use supervised algorithms for learning.

Reviewer comment (Experiments):
A comparative evaluation of the proposed method with leading NAS methods for SNNs would enhance the paper's credibility and relevance. Are there plans to include such a comparison in the future?

Author response:
Yes, the methods in "AutoSNN: Towards energy-efficient spiking neural networks" and "Neural architecture search for spiking neural networks" are very interesting. AutoSNN focuses on spike-aware NAS to find architectures that reduce spike count while maintaining accuracy, whereas NASSNN explores architectures with both forward and backward connections to enhance temporal information processing in SNNs. However, we have to point out that the network architectures used in these two works are both not RSNN, and the proposed methods are not directly applicable to RSNN. Although direct comparisons are not appropriate, we will explore the possibility of combining different methods in the future.

Thanks for your insightful question! We'd be glad to discuss any unclear details further.

Reviewer comment (Evaluation):
While the architecture found by the model outperforms the other baselines, it comes at a significant a computational cost. Please report the time taken to run the search and the number of parameters of each model.

Author response:
Yes, after we determine the optimal architecture, we will continue to use BP and IP to train on the optimal architecture for several epochs to adjust the weights. For example, for TI46-Alpha dataset, a complete training process of a RSNN with 800 neurons hidden layer for TI46-alpha dataset, including 150 epoch for cell size search, 150 epoch for connection type search and 400 epoch for finetune, takes 4 hours on single NVIDIA GeForce RTX 3090 GPU.

Reviewer comment (Reproducibility):
While the intra-motif connections are detailed, the search space of inter-motif connection is not elaborated in 3.2.1. Have I missed it? If not, can you please describe that too? Given that there are no explicit constrains in the search space formulation of the layer connection matrix, how can we enforce sparse inter-motif connections? In DARTS search space, a node i can only be connected with a node j if i < j. One can enforce a similar constraint in this case too.

Author response:
The search space of inter-motif connections are intuitively illustrated in Figure 1\&2. In detail: 1. In terms of allowable connections and the number of such connections with in a SC-ML layer, inter-motif connections are manually restricted between adjacent motifs, and neurons in one motif are only allowed to be wired up with the corresponding neurons in the neighboring motifs (The i-th motif is only allowed to be connected to the i-1 and i+1 motifs). This means that with in a SC-ML layer, the inter/intra-motif connection of n neurons in space is local and sparse, reducing from the order of $O(n^2)$ to the order of $O(n)$, which also reduces its search space. 2. In terms of connection types, we allowed the type of of an inter-motif connection to be any one of the following three: inhibitory, excitatory or non-existent; and we determined the type of each inter-motif connection through search. Furthermore, inhibitory connections are set to fixed values and non-existent connections are set to 0, neither of which needs to be optimized.

The aforementioned configuration was designed to allow flexibility in constructing a high-performance sparse spiking RNN, while simultaneously constraining the search space to ensure practical viability in solving the complex optimization problem associated with RNN architecture. The proposed optimization framework is highly flexible and can accommodate other types of sparsity constraints.

Reviewer comment (Experiments):
Can you also perform random search in your search space? In NAS, generally random search is also used as a baseline to understand the effectiveness of the proposed search algorithm.

Author response:
Please check general response/Section: Additional Information in Appendix of our paper. We provide comprehensive data & analysis of random search baseline, learning curve and ablation study of IP rule there.

Thanks for your insightful question! We'd be glad to discuss any unclear details further.