Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The authors prove 3 theorems for "self-connection" (which I understand as recurrent) spiking neural networks (scSNNs), showing that scSNNs are universal approximators, they can approximate radial functions using polynomial spiking neurons, and they can approximate multivariate spike flows within polynomial time, which they also verify via simulations.
strengths_and_weaknesses: Looking at equation (1), "self-connection" seems to mean recurrent connections. There are already theorems showing that recurrent neural networks can approximate dynamical systems (e.g. Schäfer, A. M., & Zimmermann, H. G. (2006). Recurrent Neural Networks Are Universal Approximators. https://doi.org/10.1007/11840817_66 or Seidl, D. R., & Lorenz, R. D. (1991). A structure by which a recurrent neural network can approximate a nonlinear dynamic system. https://doi.org/10.1109/IJCNN.1991.155422). The novelty to my understanding here is to show the same with recurrent *spiking* neural networks -- I am unsure if this has been shown already or is new. The authors are advised to clarify what is new compared to the non-spiking (or possibly existing spiking) proofs.

The proof uses spike response model (SRM) for the neurons. Would the proof also go through if a Leaky Integrate and Fire (LIF) model is used for neurons?

The definition of $f(\mathbf{x},t)$ in Theorem 1 seems to be a weighted sum of the instantaneous firing rates of the individual neurons as defined in equation 4. If I understand correctly, theorem 3 seems to suggest that without training, one can decode these instantaneous firing rates to approximate any MSF. This is reminescent of liquid state machine (reservoir computing). Perhaps the authors can highlight what is different and/or connections if any to the same? Also in theorem 3, $\mathbf{V}$ is an nxm matrix -- I suppose this is different from the $\mathbf{V}$ defined in equation 1 as an nxn self-connection matrix. If yes, then please use a different symbol here, else please clarify.

Disclaimer: I am not qualified to comments on the mathematical soundness, and would leave it to other reviewers.
questions: The authors should also connect self-connection SNNs with recurrent SNNs -- to my understanding these are the same and the latter term is far more prevalent.

Rest of the questions / comments are listed above. Overall the manuscript is very dense and has not been connected well to other results and concepts/practices in the field. No effort is made to make the material useful/accessible to a wider audience. If the authors can clarify my points above and provide more intuitive understanding as well, I would be willing to improve my rating.
limitations: N/A
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: In this work the authors explore spiking neural networks (SNN) with self-connections and their ability to approximate functions spatially and temporally and how computationally efficient they are. The authors show in the paper that  SNNs can approximate approximate continuous dynamical systems within polynomial parameters and time complexities.

strengths_and_weaknesses: The paper builds on previous work showing that self connections are needed in SNN to work on spatiotemporal data due to their bifurcation dynamics. The paper explores the theoretical properties of SNNs with self-connection which is a novel contribution. However, since it closely follows the previous paper, which already showed that self-connections are needed. the originality is very limited. The theoretical approach used in this paper is also not novel. The theory presented in the paper is sound and there are no major errors as far as I can understand. The paper is fairly clearly presented. However, I would suggest to improve figure 2 by making the axis labels more legible, they are way too small.  Regarding the significance of the paper, the paper essentially corroborates previous literature and therefore I think its significance is also limited. It is unlikely to make a huge impact, but the work done here was necessary and it has been solidly executed.    
questions: I have no questions for the authors.
limitations: yes.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This manuscript, theoretically studies the approximation powers and computational efficiency of spiking neural networks with self-connections (scSNNs)). It is demonstrated that the self-connection structure enables spiking neural networks to approximate continuous dynamical systems within polynomial parameters and time complexities.
strengths_and_weaknesses: **Strengths**

- The paper is ideally well supported by theoretical arguments. Some proofs (given in both the main text and Appendix) are written in a nice and technical way. Providing summarized proof ideas after theorems is very helpful. 

- The paper is well organized and clearly written. The problem was well motivated and the writing was consistent in tackling this problem throughout the work.


**Weaknesses**
- The paper needs more experimental support/evidence. It would be worthwhile to see at least one more example to evaluate the obtained theoretical results.

*Minor issues*:

-- Fig. 2 is not so visible and needs to be improved. Font size and and axes labels should perhaps be larger. 

-- It might be better to number some equations (in the proofs) so that one can refer to them.

-- There are some typos/grammatical errors: Line 525, denote ---> denotes; line 530, we an ---> we can; line 563, "**There**, it is sufficient to show [...]"? Lines 568 \& 570, is ---> be
questions: 1- Why Theorems 1 \& 2 only consider compact subsets $K \subseteq \mathbb{R}^m$? If the assumption of compactness is removed or at least weakened, can both theorems be still proven?
limitations: The authors have discussed the limitations of the work and there is no potential negative societal impact.
review_title: null

Meta-review: metareview: This paper shows that spiking neural networks with self-connections can approximate continuous dynamical systems with polynomial parameter and time complexities. All reviewers agreed that the contributions of this paper were clearly above the acceptance threshold.
confidence: Certain
Decision: Accept