Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 4: Ok but not good enough - rejection
review_content: summary: The manuscript proposes to improve a liquid state machine (implemented in spiking neurons) by adding astrocyte-inspried STDP-mechanism to automatically poise at a 'critical' state, as quantified by a branching factor. The submission claims that close to the so-called 'edge-of-chaos', the proposed networks have improved performance, as quantified in MNIST and N-MNIST.
main_review: Originality:
* liquid state machines (LSM) are a classical alternative to BPTT. Here are novel neuron-astrocyte LSM is being proposed and benchmarked, which seems to be original.

Quality:
* The presented study is a purely numerical work, a mathematical understanding of the neuron-astrocyte LSM is missing.
* The paper claims that the NASLM self-regulate towards the edge of chaos, but this claim is not being supported by standard measures of chaos (e.g. Lyapunov exponents). Instead, a "branching factor" is being computed, which is being interpreted as a proxy of chaos. This claim is not supported by any further evidence.

* Given the numerical nature of the study, important information is missing, e.g. how the baseline was optimized. Moreover, a comparison not only in terms of performance but also in terms of total computational cost (in flops or GPU hours) is missing.

Clarity: The manuscript is written clearly and very understandable. 

* significance:
The significance of the presented work seems to be limited, as the NASLM is only benchmarked against other versions of SLM, so it is not clear how it compares to more widely used RNN training methods.

limitations_and_societal_impact: The presented does not systematically vary various parameters of the proposed NASLM to investigate its impact on both chaoticity and performance so the overall claim that the performance of liquid state machines is improved closed to the edge of chaos using the proposed astrocyte-mediated mechanisms seems not fully supported.
To spell it out in more detail:
* Quantification of chaos (Lyapunov exponent) is missing
* Mathematical theory describing the self-tuning towards an 'edge-of-chaos' is missing
* Mathematical theory linking 'edge of chaos' to superior performance is also missing.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The authors propose of a way modulating the connections in a liquid state machine (LSM) so that it moves towards the edge of chaos (EOC) to maximize its performance for various tasks. They do this using by introducing an additional "astrocyte" that modulates the parameters of STDP in the neuron model -- specifically the depression term. The authors compare their model to other forms of LSMs.

main_review: **Update**: The authors response addressed most of my major concerns, and therefore I would like to upgrade my rating.

## Originality: 
Modulating the STDP parameters of a network in this particular way to move the network towards the edge of chaos is quite novel to my knowledge. The authors do an adequate discussion of related work. Being able to move the network towards EOC in an automated way using such a relatively simple mechanism is also a very nice feature to have for training liquids.

## Quality: 
The approach is well motivated and simple, and the empirical analysis is reasonable.  The authors compare their approach with a wide variety of other published architectures.

Testing the model on further standard benchmarks for e.g. with TIMIT or other speech recognition tasks can make the performance results more convincing.

There are also differences in how STDP is applied between LSM+STDP and NALSM -- in NALSM the STDP continues to adjust weights when collecting spike count samples but not in the other case. Why?

A bit more generally, what's the spiking rate of the liquid? This is a useful metric to see how efficient it could be on neuromorphic hardware and how relatable it is to biology.

A comparison to backpropagation based approaches for setting liquid weights such as [1] could also be made:

[1] Subramoney, Anand, Franz Scherr, and Wolfgang Maass. "Reservoirs learn to learn." arXiv preprint arXiv:1909.07486 (2019).

Minor: how important is it to have distance dependent connectivity?


## Clarity: 
While the text does convey the core points well, the writing and structure leave much to be desired. There are various redundant parts and some details are missing in the text. Some examples:
* eqn. 3: C is not defined.
* l.105: What does $ A_{-}^{astro} \rightarrow A_{-} $ mean?
* para starting at l.214 is very redundant with figure caption. Ditto para starting l.285


Other comments:
* l.130: The sentence reads as if all the weights in the liquid are equal.
* I would suggest describing LSM+AP-STDP at least briefly.
* Ditto for how the branching factor is calculated.
* It should be explicitly mentioned that the results in Fig. 2 are without astrocytes.
* Need more details on the kernel quality metric.

Minor:
* In many places, references are made using just the number, which could be ambiguous. For e.g. 158: 2.1 should be Sec. 2.1, l.164 etc.

## Significance: 
Being able to automatically move a liquid towards EOC can be quite important to avoid manual tuning often required for liquids. But the authors use a heuristic approach to doing this, which might limit how much it can be built on. But given the empirical results, this method can be extremely useful for practitioners.
limitations_and_societal_impact: The authors do not discuss limitations of their work.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper extends liquid state machine (LSM) with a biologically inspired astrocyte model (NALSM) to optimize the neuronal dynamics in LSM at the edge-of-chaos, to improve the accuracy and stability of LSMs. With NALSM, the spiking model can maintain a high accuracy without re-tuning parameters for different datasets. Experiments with MNIST and N-MNIST dataset demonstrate that NALSM outperforms classic LSM and other STDP-based LSM approaches, and most existing spiking neuron models including multi-layer SNNs.
main_review: 1. Overall, the idea of borrowing astrocyte mechanisms to build a self-organized dynamic controller for LSM is novel and interesting. The paper is well organized and presented. Although LSM model is currently a narrow topic, the way of thinking is inspiring and can be beneficial to other spiking neuron models. 

2. Why the spike-timing-dependent plasticity (STDP) based inter-connection learning is beneficial for LSM models?

3. Discussions and comparisons with other edge-of-chaos modulation approaches can be added to better demonstrate the advantages of NALSM.

limitations_and_societal_impact: Yes.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: Liquid State Machines (LSM)  circumvents the need for backpropagation and allows the training of recurrent neural networks by local plasticity in the readout weights. Previous studies have shown that optimal performance is achieved near a critical phase transition of the reservoir (the liquid). However, the exact parameters of the critical phase transition in the liquid vary and depend on the input. In this work, the authors present a biologically inspired model that allows self-organization near the critical transition. Here, the authors propose a biologically inspired model in which the liquid self-organizes near the critical point. The model is composed of a reservoir of integrate-and-fire neurons. The synaptic efficacies within have unsupervised spike-time dependent plasticity (STDP). The authors propose an additional dynamical variable akin to the activity of astrocytes in the circuit, which modifies the STDP learning rates. The model self-organizes close to the critical point. The authors train several networks on the MNIST and N-MNIST dataset (a spiking version of MNIST, made for neuromorphic computing). They show that the SLM model with astrocyte-dependent STDP plasticity outperforms other LSM models.
main_review: ### Impact

The main novelty in this work is suggesting that astrocytes may be responsible for the self-organizing of cortical circuits close to criticality. This is an interesting concept and may pave the way for more works studying cortical circuits as liquid state machines.

### Edge of chaos and criticality

It is not clear to me from the paper which critical transition, or edge-of-chaos criticality, the authors imply in the text.  

(I) The authors refer to the critical point as the edge-of-chaos in the liquid dynamics (see, e.g., the caption of figure 1). I do not believe that the studied critical point is the onset of chaos of the liquid state, and there are several reasons for that:

1. A spiking model was previously argued not to show a chaotic phase transition (Engelken et al). It was later shown that for spiking, the transition is smoothed to a continuous crossover and the critical effects are strongly diminished (Kadmon and Sompolinsky, PRX, 2015)
2. The connectivity model in eq (3), in which neurons are mostly connected to their neighbors is not known to show chaotic behavior (as far as I know). 
3. Strong spatial variance is needed to induce chaos, but it is not clear what the parameter 'C'  is in eq (3). In general, for networks that respect Dale's law, in order to get chaotic behavior, there needs to be a strong balance between excitatory and inhibitory populations [Van Vreeswijk and Sompolinsky 1998]. Alternatively, recent work has studied chaotic dynamics in heavy-tailed distribution [Kuśmierz et al., PRL, 2020]. The authors show neither in their model.
4. There is no evidence that the transition in question is the transition to chaotic activity. The authors use a form of a branching factor (BF) as the order parameter (more on that below). BF>1 leads to critical excitability, and not chaotic behavior.

(II) The authors refer to [22-24] as evidence of edge-of-chaos computation in the brain.  However, these works consider a critical-branching phase transition which is associated with the Mean-Field Directed Percolation (MF-DP) universality class. This is not the transition to chaos (unless in specific settings when the two coincide [see Kuśmierz et al. 2019]. The critical branching-criticality is defined when the branching factor is (BF=1), as implied in this paper. However, it is defined when a single spike (in the liquid!) elicit one spike on average (in the liquid!). The branching factor defined here is different (see my comment below). 

(III) Finally the authors may be referring to the edge-of-chaos in feedforward networks, as studied in [Hayou et al. (2019); Lee et al. (2018); Schoenholz et al. (2017)] for rate networks. However, this is not a critical point of the liquid and is not associated with improved computation shown in [19-21]. Furthermore, the edge-of-chaos in the feedforward networks mentioned is defined by the spectrum of the Jacobian of the connectivity matrix, which is not studied in the current paper, so the relation to it is not clear if it exists.

### The use of the branching factor

The order parameter used to measure the dynamical transition in the work is the branching factor (BF). In the proposed model, the dynamics of the astrocytes are driven by a proxy for this measure. There are a few problems I find in this formalism:

1. The criticality in [22-24] is defined with a branching factor of 1 when each spike in the liquid elicits one spike on average within the liquid. This is a different point than the information propagation in feedforward networks [e.g., Hayou et al. (2019); Lee et al. (2018); Schoenholz et al. (2017)]
2. Equations (7) and (8) are not normalized to the size of the input and liquid populations. In the experimental results, the authors change the fraction of liquid neurons that are connected to the astrocyte. Unless I misunderstood, this should change the measure of BFproxy and the astrocyte dynamics. More importantly, is not clear what is the meaning of BF=1 in the case when there is no normalization to the number of neurons sampled.


### Novelty
The paper emphasizes two results: that computation near criticality is better and that astrocyte is a way to get self-organized criticality (SOC). The first was discussed extensively in the past [19-21]. As for SOC, there have been numerous papers suggesting different schemes [e.g., Girardi-Schappo et al, Physical Review Research 2020]. While the idea of using astrocytes as a way to get SOM is interesting, it is not conceptually new in terms of computation, and I feel that this work will fit better in a biologically oriented venue.
limitations_and_societal_impact: yes
review_title: null

Meta-review: The reviewers agree that the main contribution of this paper, i.e. that the hypothesis that astrocytes help networks to self-organize close to criticality, is novel. However, the concrete impact of this contribution is somewhat unclear. Is this a statement about the function of astrocytes in biological neural circuits? If so, the paper would ideally argue in greater detail that their astrocytes model is realistic, and furthermore that other spike-rate adaptation mechanisms do not have similar, beneficial effects. Conversely, as a contribution to algorithms on neuromorphic hardware for tackling practical problems, this work would benefit from additional comparisons to alternatives for introducing stabilizing negative feedback mechanisms. Therefore, the overall contribution of this paper is below acceptance.


Decision: Accept (Poster)