Increasing Liquid State Machine Performance with Edge-of-Chaos Dynamics Organized by Astrocyte-modulated Plasticity
Authors: Vladimir A Ivanov
Konstantinos P. Michmizos
Venue: NeurIPS 2021 Poster (2021)
Keywords: liquid state machine
astrocytes
spiking neural networks
neuromorphic computing
neuron-astrocyte networks
STDP
self organized criticality

Abstract: The liquid state machine (LSM) combines low training complexity and biological plausibility, which has made it an attractive machine learning framework for edge and neuromorphic computing paradigms. Originally proposed as a model of brain computation, the LSM tunes its internal weights without backpropagation of gradients, which results in lower performance compared to multi-layer neural networks. Recent findings in neuroscience suggest that astrocytes, a long-neglected non-neuronal brain cell, modulate synaptic plasticity and brain dynamics, tuning brain networks to the vicinity of the computationally optimal critical phase transition between order and chaos. Inspired by this disruptive understanding of how brain networks self-tune, we propose the neuron-astrocyte liquid state machine (NALSM) that addresses under-performance through self-organized near-critical dynamics. Similar to its biological counterpart, the astrocyte model integrates neuronal activity and provides global feedback to spike-timing-dependent plasticity (STDP), which self-organizes NALSM dynamics around a critical branching factor that is associated with the edge-of-chaos. We demonstrate that NALSM achieves state-of-the-art accuracy versus comparable LSM methods, without the need for data-specific hand-tuning. With a top accuracy of $97.61\%$ on MNIST, $97.51\%$ on N-MNIST, and $85.84\%$ on Fashion-MNIST, NALSM achieved comparable performance to current fully-connected multi-layer spiking neural networks trained via backpropagation. Our findings suggest that the further development of brain-inspired machine learning methods has the potential to reach the performance of deep learning, with the added benefits of supporting robust and energy-efficient neuromorphic computing on the edge.

Increasing Liquid State Machine Performance with Edge-of-Chaos Dynamics Organized by Astrocyte-modulated Plasticity
Vladimir A. Ivanov Department of Computer Science
Konstantinos P. Michmizos Department of Computer Science
Rutgers University Piscataway, NJ vladimir. www.:ivvvvvvrters..... edu
Rutgers University Piscataway, NJ mi michmizos@cs rutgers.....r.............. edu
Abstract
The liquid state machine (LSM) combines low training complexity and biologi- cal plausibility, which has made it an attractive machine learning framework for edge and neuromorphic computing paradigms. Originally proposed as model of brain computation, the LSM tunes its internal weights without backpropagation of gradients, which results in lower performance compared to multi-layer neural networks. Recent findings in neuroscience suggest that astrocytes, long-neglected non-neuronal brain cell. modulate synaptic plasticity and brain dynamics, tuning brain networks to the vicinity of the computationally optimal critical phase tran- sition between order and chaos. Inspired by this disruptive understanding of how brain networks self-tune, we propose the neuron-astrocytte liquid state machine (NALSM that addresses under-performance through self-organized near-critical dynamics. Similar to its biological counterpart, the astrocyte model integrates neuronal activity and provides global feedback spike-timing-dependees plasticity (STDP), which self-organizes NALSM dynamics around a critical branching factor that is associated with the edge-of-chaos. We demonstrate that NALSM achieves state-of-the-art accuracy versus comparable LSM methods, without the need for data-specific Hand-tuning. With top accuracy of 97.61 on MNIST, 97.51% on N-MNIST, and 85.84% on Mushion-MNIST.. NALSM achieved comparable performance to current fully-connected multi-layer spiking neural networks trained via backpropagation. Our findings suggest that the further development of brain- inspired machine learning methods has the potential to reach the performance of deep learning, with the added benefits of supporting robust and energy efffiieef neuromorphic computing on edge.
Introduction
With the recent rise of neuromorphic [1H4] and edge computing 5 6 the liquid state machine (LSM) learning framework (7) has become an attractive alternative 88-1 to deep neural networks owing to its compatibility with energy-efficient neuromorphic hardware 12 14 and inherently low training complexity. Originally proposed as a biologically plausible model of learning, LSMs avoid training via backpropagation by using sparse, recurrent, spiking neural network (liquid) with fixed synaptic connection weights to project inputs into high dimensional space from which a single neural layer learn the correct outputs. Yet, these advantages sub-par accuracy and 2) extensive data-specific hand hand-tuning of liquid weights. Interestingly, these
deep networks come at the expense of 1)
Coode and data available at htttts://github... httpp:://gituub..c///ommaa wwt./combaa-aaaa///A.....
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

two limitations have been targeted by several studies that tackle one [13]] 16 or the other [7 18] but not both. This has limited the widespread use of LSMs in real-worls applications [8. In that sense, there is an unmet need for a unified, brain-inspired approach that is directly applicable to the emerging neuromorphic and edge computing technologies, facilitating them to go mainstream.
As general heuristic, LSM accuracy is maximized when LSM dynamics are positioned at the edge- of-chaos 19-21 and specifically in the vicinity of a critical phase transition [22/25] that separates: 1) the sub-critical phase, where network activity decays, and 2) the super-critical (chaotic) phase, where network activity gets exponentially amplified. Strikingly, brain networks have also been found to operate near a critical phase transition 28) that is modeled as branching process [25] 26). Current LSM tuning methods organize network dynamics at the critical branching factor by adding forward and backward communication channels on top of the liquid [T3] [6]. This, however, results in significant increases in training complexity and violates the LSM's brain-inspired self-organization principles. For example, these methods lack local plasticity rules that are widely observed in the brain and considered a key component for both biological 29] and neuromorphic learning 3 2 A particular local learning rule,
plasticity (STDP), is known to improve LSM accuracy [7 [8]. Yet, current methods of incorporating STDP into LSMs further exacerbate the limitations of data-specific hand-tuning as they require additional mechanisms to compensate for the This signifies the scarcity of LSM tuning methods that are both computationally efficient and data inta-indeeendent.
STDP-imposes saturation of synaptic weights [1 30 33
long-negleen negleeted non-neuronal cell in the brain, astrocytes, is now known to play key roles in modu- lating brain networks [34|39] from modifying synaptic plasticity 42 to facilitating switching
between cognitive states 43 46 that have been linked to a narrow spectrum of dynamics around the
critical phase transition 47-51 The mechanisms that astrocytes use to modulate neurons include
the integration of the activity of thousands of synapses into a slow intracellular continuous signal that feeds back to neurons by affecting their synaptic plasticity (52 42). The unique spatio- temporal attributes [56]] [5] identified in astrocytes align well with the brain's 's remarkable ability to self-organize its massive and highly recursive networks near criticality. That is why astrocytes present a fascinating possibility of forming a unified feedback modulation mechanism required to improve baseline LSM accuracy while eliminating data-specific hand-tuning"
Here, we propose the neuron-astrocyte liquid state machine (NALSM), where a biologically inspired astrocyte model organized liquid dynamics near a critical phase transition, by modulating STDP. We show that NALSM combined the computational benefits of both STDP and critical branching dynamics by demonstrating its accuracy advantage compared to other LSM methods on two datasets: 1) MNIST (58), and 2) N-MNIST [59]. We demonstrate that, similar to its biological counterpart that handles new and unstructured information with robustness and versatility, NALSM maintains the state-of-the-art LSS performance without re-tuning training parameters for each tested dataset. We also show that NALSM with large enough liquid can attain comparable accuracy to fully fully-connected multi-layer spiking neural networks trained via backpropagation on 1) MNIST (58), 2) N-MNIST [59. as well as 3) Fashion-MNIST [60]. Our results suggest that the under- merfo-perrfomance and high training difficulty of current neuromorphic methods can be addressed by harvesting neuroscience knowledge and further translating biological principles to computational mechanisms.
2 Methods
2.1 The neuron-astrocyte liquid state machine
To construct the NALSM, we started with baseline LSM model consisting of 2 layers: 1) spiking liquid, and 2) a linear output layer. Next, we added STDP to the LSM liquid, forming the LSM+STDP model. We developed biologically faithful which we embedded in the LSM+STDP liquid, to form the NALSM. The process is formalized below.
(LIM) astrocyte model,
LSM Model We implemented the baseline LSM a 3-dimensional neural network (liquid) con- sisting of 1, 000 neurons surrounded by 1-dimensional layers of input and output neurons. Number of input neurons was 784 and 2, 312 for MNIST and N-MNIST, respectively (See Appendix A.I). We used the leaky-integrate-and-frre (LIFF) model 3 for input and liquid neurons, modeled as:
du: =
(t(() + (t) - (tax(t)
(1)
dt To
2

---

(t(t) Σ j7t
wiylas * at)((t) +
(2)
where vi is the membrane potential and u, is the synaptic response current neuron i, 0, the membrane potential threshold, o; a;(() = Σx 5(1 -(t--1 ( is the spike train of neuron i with 1 being the time of the k-th spike, Wij is the weight connecting neuron j to i, by the bias of neuron and (t) Tu (xeepptt///)/l (t) is the synaptic filter with H(t) being the unit step function (See Appendix A. All LIF neurons had 2 ms absolute refractory period. Liquid neurons were excitatory and inhibitory with 80% ratio. Input neurons did not have an exciratory/nhiiitory distinction and had random excitatory and inhibitory connections to liquid neurons. From here on, we will refer to connections between input neurons and liquid neurons as II. connections, inter-liquid connections as III,, and liquid to output connections as L.. In line with 7 we created LLI connections using probabilities based on Euclidean distance, (i,,j), between any two neurons i,
P(i.j) C eap (- (-(Bii))) 2
(3)
with closer neurons having higher connection probability. Parameters C' and À set the amplitude and horizontal shift, respectively, the probability distribution (See Appendix A.3). Density of IL connections was 15%. The output layer was dense layer consisting of 10 linear neurons.
LSM+STDP Model We added unsupervised, local learning to the LSM model by letting STDP change each LL and IL connection [61], modeled as:
dw dt
A pree Σ (ppst) - Tpost 5(t - t(pre)
(4)
where A+ A_ 0.15 are the potentiattio//eeprssioo learning rates and Pprelfppor post are the pre/post-synaptic trace variables, modeled as,
dT dTppe pre
-T Tpre + a+ Σ
8(t ppre)
(5)
dt
dT post
-post + a Σo(t ö(t otposs)
(6)
dt
where a4 0.1 are the discrete contributions of each spike to the trace variable, 7 r 10
the decay time constants, tiree and foost the times of the pre-synaptic and post-synaptic
spikes, respectively. We constrained connection weights to: 1) IL: [-3. 3] 2) excitatory LL: [0. 3),
and 3) inhibitory LL: [-3, 0). We used the same STDP parameters for all models and experiments.
LIM Astrocyte Model We developed the astrocyte model as leaky integrator with continuous output value Austro expressed as:
dsastra
Tasto
-Austro Austro
+ Wastro
8(t - f.,) - Wastro
Σ jEN
8(t - tz) bastro
dt
NENo.
where Austro directly mapped to A_ in equation (4 bastro = A+ adjusted the astrocyte output to the fixed STDP potentiation learning rate, Nit and Niup are the sets liquid and input neurons, respectively, and Wastro set astrocyte responsiveness to network activity (See Appendix A.4), Ignoring the decay and bias terms, the astrocyte model computed the difference in the number of spikes produced by liquid neurons and input neurons. Functionally, this is equivalent to computing the ratio of spikes emitted by the liquid over the input neurons:
BF prrony (1) EiENi, 5(1 - 1,))
(8)
Ejenuo 8(t 1;)
Specifically, when the liquid produced more spikes than the input neurons, their difference was positive which translated to Fprozy > 1.0, and vice versa. This approach to measure liquid dynamics acted as network level approximation of the branching factor, dBF, which is normally
3

---

A
Neuron Astrocyte Liquid
B
Astrocyte enforces critical branching factor (BF)
STDP Depression Leumiing Rate
$ STDP Potzntiation Learning Rate
STDP
STDP
Depression Leurning
Potemiation
Leurning
Inputs
3D Liquid
Outputs
Rate
Rate
Weights
Weights decrease
BP increases
liquid decresses
1.5
Branching Factor
1.0
Computational Units
Connections
Modulatory Units
Spiking input nearon
Expilatery
vin STDP
1rr Astrccyic
0.5
polynomial
Spiking caciistory neuron
Inhibitory
Global modulation TTD depecssion
0.5
1.0
1.5
2.0
Spiking inhibitcey
truine via STDP
Linear oupput neuron
gradient deseent
Trained via
learnin mile
BF approximation estimated by astrocyte
Figure I: NALSM architecture and astrocyte modulation of liquid dynamics. ( The neuron- astrocyte liquid was modeled as a 3-dimensional network of excitatory and inhibitory spiking neurons connected with sparse, recurrent connections with spike-timing-dee degendennd plasticity (STDP). Input neurons projected excitatory and inhibitory connections to the liquid. Receiving each liquid neuron's spike count per input sample, dense linear output layer was trained via gradient descent to classify inputs. B To organize liquid dynamics the critical branching factor, an astrocyte integrated input and liquid neuron activity and, in turn, set the global STDP depression learning rate. Data points are binned averages over 'BF approximation" metric. Error bars are standard deviation. See Appendix A. for polynomial fits.
evaluated for each neuron (See 2.3.1 We empirically confirmed that BFprrry = 1.0 aligned with the critical branching factor, OBF 1.0 (Fig. B). Hence, as dynamics became progressively super- critical BØ 1.0), BFprrry became greater than which caused the LIM astrocyte increase STDP depression learning rate above the fixed STDP potentiation learning rate (Aastro - A_ A+). This caused STDP to decrease the average weight of LL and IL connections, which decreased number of spikes produced by the liquid and made dynamics less super-critical (Fig. B). The reverse occurred as dynamics became progressively sub-critical. As a result of astrocyte modulation, liquid dynamics oscillated between sub-critical and super-critical until eventual stabilization near the critical branching factor (See Appendix A.4).
NALSM Model We completed NALSM by adding the LIM astrocyte the LSM+STDP model's liquid (Fig. A].. As described above, the LIM astrocyte integrated activity from input and liquid neurons, and continuously controlled the STDP depression learning rate.
2.2 Training
Model training was done in steps: 1) initialization IL and LL liquid connections, 2) passing all data through the liquid resulting in liquid neuron spike counts, and 3) training the output layer on the spike counts. The steps are further detailed below.
2.2.1 Liquid initialization
LSM We initialized all IL and LL connections with single weight value, maintaining originally defined connection signs 622 Weights were constant during spike count collection.
LSM+STDP We initialized IL and LL connections with STDP by consecutively presenting all the training images to the liquid. Starting with initially maximal connections, 3(-3) for exci- (anhbi(inhi connections, we let STDP continuously adjusted weights while presenting the liquid with a randomly ordered series of MNIST training image snapshots, each lasting 20 ms. For N-MNIST, we randomly sampled each 20 ms snapshot from the 0 250 ms range, as a way to account for the variability in the temporal dimension (See 2.3) In each case, we used a total of 50, 000 snapshots, each corresponding to unique training image, We used STDP only for weight initialization. Initialized weights were fixed during spike count collection.

---

NALSM We used the LSM+STDP weight initialization process with the exception of added STDP modulation by the LIM astrocyte. We used this set of initialized weights as the starting point for each sample in the spike counting phase, during which synaptic weights to compensate for slight deviations in dynamics caused by each input sample's different level of activity. For each sample, parameters A+ from 4 and bustro from ( were both initialized to 0.15 and decayed at a rate of 0.99 for the duration of sample input.
STDP continued to adjust
2.2.2 Output layer training
We assembled spike counts by presenting each sample image to the liquid for 250 ms and counting the number of spikes emitted by each liquid neuron for the full duration of input. We used Adam optimizer to batch train the output layer spike count vectors by minimizing the cross entropy loss with L--rregularization,
Areg |Woutl 2m
C(ye, 0) = m Σ ( + (1 - 3/s Mog - 9s) +
(9)
where m = 250 is the batch size, Wont is the output layer weight matrix, Areg = 5 x 10 10-10pp the regularization hyperparameter, y/ and ÿs are the normalized vectors denoting the predicted label and the target label, respectively. Prior to training, we initialized output layer weights/biases to 0.0, and the learning rate to 0.1. We trained the output layer until validation accuracy peaked (up to maximum of 5, 000 epochs), at which point we evaluated model test accuracy.
2.3 Experiments
We performed all LSM comparison experiments on MNIST and N-MNIST datasets (See A.1). Using 10 randomly generated networks for each dataset, we trained 1) the baseline LSM model, 2) the LSM+STDP model, 3) the NALSM model (See and 4) the LSM+AP-STDD model, a method for incorporating STDP the LSM liquid [77] (See Appendix A.5). First, we evaluated LSM model accuracy with respect to liquid weight, which ranged in 0.4 - 1.2 for MNIST and 0.8 - 1.35 for N-MNIST. We used a random seed for each training session. Next, we evaluated corresponding network dynamics of each network/weight combination by measuring the liquid's branching factor on 20 randomly sampled inputs (See To have comparable results for each network, we trained the remaining models using the same seed that resulted in peak LSM accuracy. For NALSM, used the same initialization and parameters for all networks and datasets. For LSM+AP-STDE we hand-tuned STDP control parameters for each network and dataset combination maximize validation accuracy (See Appendix A.5). Additionally, we trained NALSM on Fashion-MNIST dataset (See A.1) using the same 10 randomly generated networks that we had used for MNIST.
Sparse neuron-astrocyte connectivity We tested NALSM's accuracy as a function of neuron- astrocyte connection density on 3 best performing networks (per dataset). Keeping the proportion of neurons sampled by the astrocyte the same for both input neurons and liquid neurons, we trained NALSM with 10%, 20%, 40%, 60%, and 80% neuron-astrocys density over 3 seeds for each of 3 networks. Regardless of connection sparseness, all IL/LL connections were modulated by Aastro (See 2.
NALSM with larger liquid sizes We tested NALSM performance for larger liquids. For each size, we trained randomly generated networks, each on a random seed. All parameters and initialization were same as for 1, 000 neuron liquid. For maximum accuracy, we trained NALSM with an 8, 000 neuron liquid. For each dataset, we used randomly generated networks trained on a random seed. Parameter Wastro 0.0075 for all datasets. All other parameters and initialization were before.
2.3.1 Branching factor of liquid
To evaluate a liquid's dynamics, we used the network branching factor, OBF, which quantifies network information flow amplification/decay Liquid dynamics are sub-critical, near-critical and super-critical when oFF < 1.0, OBF ss 1.0, and oFF > 1.0. We calculated og as done (30] with offset é = 0 and time window A ms per (26].
5

---

2.3.2 Kernel quality of liquid
We evaluated liquid 1) linear separation, and 2) generalization capability using methods from 63 For the MNIST and N-MNIST test sets, we computed the rank of matrix M assembled from randomly selected spike count vectors, resulting in shape Mig x k. We repeated this on 1, 000 shuffles of spike vectors. For linear separation, we used spike counts from model testing phase. For generalization capability, we added noise to input data and evaluated new spike count vectors (See 2.2.2). For MNIST, we added N (0, 125) noise to each pixel value. For N-MNIST, we time-shifted each event by (O, 10). Taken together across all models and both datasets, rescaled ranks of each measure to 0 range and subtracted measure from measure 2 as [63] Due to negative differences, again rescaled all differences to 0 1.
3 Results
3.1 Baseline LSM performance
We established a benchmark accuracy for the baseline LSM on MNIST and N-MNIST datasets (See We acquired our baseline by averaging over 10 randomly generated liquids with 1, 000 neurons
2.3
(See 2.1 The LSM achieved a top accuracy of 95. 44% (95.30 + 0.11%) on MNIST, and 95.35% 195.02 0. 1.15%) on N-MNIST (See (S2)...2). For MNIST, this was comparable to the previously reported state-of of the-the-art art LSM accuracy (64] using the same sized liquid. Further, LSM accuracy was very sensitive to the liquid's weight (Fig. 2 A).
3.2 LSM performance peaked at the critical branching factor
The peak LSM aceuracy on each dataset corresponded a different liquid synaptic weight. Specifi- cally, there were cases where liquid with weights tuned for maximum accuracy on MNIST, would catastrophically fail on N-MNIST (Fig. A). Also, LSM accuracy on MNIST plateaued for wider
A
B
C
MNIST
Accuracy on MNIST
1.4
95% -
Branching Factor
98%
1.0
92%
Near Netr-optimal liquid weight
95%
depends un dataset
polynemial R
Near optinal perfomance actiical brnnching
N-MNIST
113%
Accuracy on N-MNIST
1.4
94%
Branching Factor
96%
+
90%
1.0
95%
0.9
1.1
60%
0.6
1.0
1.4
0.6
1.0
1.4
0.6
1.0
Liquid Synaptic Weight
Liquid Branching Factor
Liquid Synuptic Weight
NALSM
LSM-AP-STDP
LSM
LSM-STDP
Figure 2: LSM accuracy depended liquid weight and dynamics. (A L.M accuracy shown as a function of its liquid weight, averaged over a set of 10 randomly generated networks for MNIST and N-MNIST datasets. BB] LSM accuracy shown with respect liquid dynamies set by liquid synaptic weight. For each weight, liquid dynamics were measured and averaged over all 10 networks. Similarly, accuracy and resulting liquid dynamics are shown for each model: NALSM, LSM+AP-STDP. and 3) LSM+STDP. ( C Liquid dynamics shown with respect to liquid weight, averaged over all 10 networks, Error bars are standard deviation. See Appendix A.10 for polynomial fits.

---

range of weights than on N-MNIST, which can be attributed to N-MNIST's greater difficulty caused by its variability over the temporal dimension (See Appendix A.1). Taken together, this indicated that LSM training requires extensive hand-tuning of weights for each specific dataset.
Since critical dynamics are well known to result in near-maximum LSM performance 65 19
-atasct-specific hand-tuning can be significantly reduced by replacing accuracy with liquid dynamics as the target output of weight tuning. Indeed, LSM accuracy was near-maximum for both datasets, when the liquid's s branching factor was in 1.0 - 1.2 range, or slightly super-critical (Fig. 2 B) (See 2.3.I). This agrees with studies showing that information transfer in finite sized systems peaks at slightly super- super-critical dynamics [66]. Although each dataset still had different weight ranges corresponding to the critical branching factor, the relationship between liquid dynamics and weight was positive for both datasets (Fig. C). Known to generalize beyond specific datasets [20), this relationship suggested that near-critical dynamics can be organized using STDP, by providing directional feedback from current liquid dynamics.
3.3
Astrocyte-modulated plasticity organized liquid dynamics near criticality
The LIM astrocyte model stabilized liquid dynamics near the critical branching factor as we presented a continuous stream of samples to the neuron-astroeyy liquid (Fig. 2 B). The NALSM's slightly super-critical stabilization suggested that liquid dynamics were the edge-of-chaos. While chaotic activity is known to correspond to super-critical branching dynamics in some cases [25], such correspondence is not guaranteed. Hence, we examined additional network properties that are necessary and indicative of chaotic activity (See Appendix A.6). Specifically, the astrocyte-modullted liquid had coexistence of small and large synaptic weights (Fig. S2), as well as a balance of excitation and inhibition, both of which are necessary for the existence of chaotic network activity [67] 68). Further supporting a chaotic activity, the neuron- astrocyte liquid spike activity appeared irregular (Fig. S3). We also performed autocorrelation analysis on liquid neuron spike trains, which further suggested the existence of chaotic activity with a correspondence between edge-of-chaos dynamics and critical branching dynamies (Fig. S4) 66 70 (See Appendix A.6). Given that liquid dynamics were directly approximated by the LIM astrocyte, which directly controlled the STDP depression learning rate (See 2.1), NALSM required dataset-specific hand-tuning. As result, we used the same weight initialization and parameters to benchmark NALSM (See
A
Accuracy on MNIST
Accuracy on N-MNIST
B
Kereel Quality of Liquid
96%
0.86
Kornel Qaulity
Accuracy
0.82
95%
0.4
89%
0.2
87%
NALSM
LSM+ AP-STDP
LSM
LSM +
NALSM
LSM + SP-STDP
LSM
LSM
NALSM
LSM+
AP-STDP
LSM
LSM +
STDP
STDP
STDP
Figure 3: Comparison of model accuracy and liquid computational capacity. (A) Accuracy
performance of the proposed NALSM model was compared, MNIST and N-MNIST, against 3 related models: 1) the baseline LSM, 2) LSM with activity- STDP (LSM+AP-STDP), and 3) LSM with unregulated STDP (LSM+STDP). For each dataset and model, accuracy was evaluated using 10 randomly generated networks, each of which was trained on a random seed. This set of 10 seeds was used for all models. ( Computational capacity of each model was measured using a kernel quality metric that encompassed the linear separation and generalization capability of the liquid (See2 2.3.2. Error bars are standard deviation.
7

---

3.4 Benchmarking NALSM performance on MNIST and N-MNIST
On both datasets, NALSM achieved superior performance to comparable LSM models of the same size. Using 1, 000 liquid neurons, NALSM achieved a top accuracy of 96.15% (95.96 = 0.13%) on MNIST and 96.13% (95.90 + 0.16%) on N-MNIST: outperforming LSM model's top accuracy by 0.71% on MNIST and 0.78% on N-MNIST (Fig.
3 A). We also compared NAL.M to state- of-the-art LSM STDP method, AP-STDP (See 2.3 The LSM+AP-STIP model required more extensive dataset-specifie hand-tuning than the baseline LSM due to its additional STDP control parameters (See Appendix A.5). Resulting in top accuracy of 95.62 62% (95.49 + 0.09%) and 95.43% (95.23 = 0. 16%). the LSM+AP-STDD model was superseded by NALSM by 0.53% and 0.70% on MNIST and N-MNIST, respectively. As a control measure, we also trained a LSM with unregulated STDP (See 2.T The LSM+STDP model significantly under-performed compared to all other models achieving a top accuracy of 90.52% (89.47 = 0.45%) on MNIST and 87. 87.71% (86.80 = 0.51%) on to the LSM+STDP liquid's 's excessive super-critical dynamics (Fig. B) which are well known to decrease liquid computational capacity [65 [66]
N-MNIST. We attributed this
The NALSM had the most robust accuracy performance across the two datasets out of all the compared LSM models. With no dataset- specific tuning. NALSM's average accuracy on N-MNIST was lower than the accuracy on MNIST by only -0. -0.05%. This was 5 50 times less than for the LSM+AP-STDP (-0.26%) LSM (-0.29%), and LSM+STDP (-2. 2.66%) models.
We attributed the NALSM's performance advantage to the improved computational properties of its liquid. For both tested datasets, the NALSM achieved slightly super-critical critical branching dynamics where baseline LSM performance peaked right before it started to decline with increasing super- critical dynamics (Fig. B). This suggested that NALSM's performance advantage, compared to a LSM with similar dynamics, was due to the addition of astrocytc eetrocyle-modulated STDP (See 2.2.1p While LSM+AP- STDP and LSM+STDP models also had STDP, their lower performance can be explained by their excessively sub-critical and super-critical dynamics, respectively (Fig. B). We further confirmed that NALSM's increased performance resulted from the improved computational properties of its liquid by measuring each model's liquid kernel quality. This encompassed both the linear separation and generalization capability of the liquid (See 2.3.2). Higher model accuracy corresponded to higher kernel quality for all models (Fig. 3 B). This is a further indication that near-critical dynamics and astrocyte-moot modulated STDP contributed to the NALSM's performance increase.
3.5 NALSM maintained performance with sparse neuron-astrocyte connectivity
The NALSM maintained its accuracy advantage even with neuron-astrocyte connection densities as low as 10% (Fig.
4p. In the brain, astrocytes contact only approximately 65% of all synapses in their
surroundings 7
We tested NALSM performance as a function of neuron-astrocyte connection
density (See 2.3 The NALSM mean accuracy decreased marginally with increasingly sparse
connectivity, while variability performance was minimal across densities. At 10% connectivity, average NALSM accuracy decreased by 0.36% for MNIST and 0. 1414 for N-MNIST compared to 100% connection density. In both cases, average NALSM accuracy was still above LSM and LSM+APSTDP average accuracy.
MNIST
N-MNIST
Accuracy
96%
NALSM
LSMIAP-STO
95%
LSM
0.8% Neuron-Astrocyte Connection Density
0.4%
0.4%
0.8%
polynomial fit
Neuron-Astrocyte Connection Density
Figure 4: NALSM maintains accuracy advantage with sparse neuron-astrocyte connectivity. For MNIST and N-MNIST, NALSM accuracy was evaluated with respect to neuron-astrocyte connection density. For each density, NALSM performance was compared to LSM and LSM+AP- STDP average accuracy. NALSM data points are average values over 9 experiments (3 networks x 3 seeds). Error bars and shaded areas are standard deviation. See Appendix A.10 for polynomial fits.
8

---

Table 1: Comparison to brain insn-ins and fully-connected multi-layer spiking neural networks.
Model
Layers
Learning Method
Accuracy
Dataset: MNIST
Unsupervised-SNT
75
STDP
95%
Multi-liquid LSM 64
2
GD on last layer
95.5%
NALSMI000
astro-STDP, GD on last layer
96.15%
LIF-BA [73
3
Broadcast feedback alignment
97.09%
Temporal SNN 76
Temporal backpropagation
97.2%
STiDi-BP 7
2
Backpropagation
97.4%
NALSM8000
astro-STDP, GD on last layer
97.61 %
SN (78
3
Backpropagation
97.93%
GLSNN 72
Global feedback alignment, STDP
98.62%
Balance-SNN 74
2
Equi-prop. STDP, STP Backpropagation Spatial and temporal backpropagation
98.64%
BPSNN 79
3
98.88%
STBP 80
98.89%
Dataset: N-MNIST
DECOLLE 81
3
Backpropagation
96%
NALSMI000
astro-STDP, GD on last layer
96.13%
AER-SNN (82)
2
Backpropagation
96.3%
NALSM8000
astro-STDP, GD on last layer
97.51 %
BPSNN 779
3
Backpropagation
98.74%
STBP [0)
Spatial and temporal backpropagation
98.78%
SLAYER 83
3
Backpropagation
98.89%
Dataset: Fashion-MNIST
VPSNN 84
2
Equi-prog prop, STDP astro-STDP, GD on last layer
82.69%
NALSMI000
83.54 %
Unsupervised-SNN 85
2
STDP astro-STDP, GD on last layer
85.31% %
NALSM8800
85.84%
BS4NN [86
2
Temporal backpropagation
87.3%
GLSNN 722
Global feedback alignment, STDP
89.05%
*GD: gradient descent
3.6 Larger liquids increased NALSM accuracy
The NALSM accuracy improved with increased liquid size, saturating at approximately 8, 000 neurons (See Appendix A. A.7). NALSM8000 achieved a top accuracy of 97.61% (97.49 + 0.11%) on MNIST, 97.51 % 97.4222666 + 0.07%) on N-MNIST, and 85.84% (85.61 + 0. 18%) on Fashion-MNIST. Compared to previously reported benchmarks on MNIST and Fashion- the NALSM8000 outperformed all brain-inspired learning methods that do not use backpropagation of gradients its approximation through feedback alignment [72, (3), with the exception of [74) for MNIST. While 74 demonstrated that fully-connected 2-layer spiking network can achieve high accuracy through a combination of biologically-plausibe plasticity rules, it not clear how such an approach would scale to more layers without some form of backpropagation. Conversely, multi-layered LSMs have been shown to work without backpropagation 1/3 (= 199, 407 453) of number of trainable(plastic) connections in [74]. Compared to top accuracies reported for fully-connected multi-layered spiking neural networks trained with backpropagation, the NALSMSON achieved comparable performance on all datasets; outperforming multiple reported results on MNIST and N-MNIST (Table T (See Appendix A.8).
[0). Further, NALSMS000 used approximately
Discussion and Broader Impact
Ironically, LSMs are one of the most brain-like and at the same time one of the most difficult train learning models. Here, we proposed an astrocyte model that merged critical branching dynamics and STDP into a single liquid, thereby simultaneously improving LSM performance and decreasing data-specifie tuning. We showed that the synergy of STDP and near-critical branching dynamics

---

improved the computational capacity of the liquid, which translated to better than tfateoo--the-ar LSM accuracy on MNIST and N-MNIST, and do so with minimal added computational cost (See Appendix A.9). Our results indicate that, given a large enough liquid, NAL.SM performance compares to current fully-connected multi-layer spiking neural networks trained via backpropagation.
The reported narrowing of the performance gap between brain-inspired LSM and deep networks suggests that studying the interaction among the brain's is computational principles can help our learning models to reach human-like performance. Indeed, our results demonstrate that the synergy of brain- inspired astrocyte- astrocyte-modulltted modulated STDP and near-critical dynamics resulted in the superior performance of NALSM compared to 1) a LSM with critical dynamics but without STDP, and 2) LSM with STDP, but without critical dynamics. Aligning with other studies showing that liquid topology impacts LSM accuracy [87]. we also showed that a brain-inspired sparse, 3D-distance-basee network architecture can improve the computational capacity of a single liquid. Specifically, our baseline 3D LSM achieved comparable accuracy to the multi-liquid LSM [64], which improved performance of a single dimensionless liquid by partitioning into multiple liquids. While we demonstrated NALSM performance using only the 3D-distance-baad based network architecture, our proposed astrocyte modulation method does not depend on network topology and, therefore, is applicable to other types of topology. In fact, our approach is also extendable to multi-liquid architectures and other local plasticity rules that follow STDP's separation of potentiation and depression components.
The astrocyte-men astrocyte-modulated modulated LSM learning framework is also compatible with the emerging neuromor- phic hardware. This is because the gradient descent that we used for training the linear output can be replaced by a single-layer spike based learning rule [88-99) 99) This makes NALSM compatible with neuromorphic hardware, exploiting in full its advantages 91.. For example, our method can leverage even further the energy efficiency of neuromorphic chips, by virtue of its low spiking rates. In line with biological ranges (92). NALSM had spiking rates that ranged from 12 H to 37 22,, depending on the input sample. These rates can be reduced further, by modifying input encoding, since liquid spiking rates are directly adjusted by the astrocyte based on input spiking rates (Fig. D
Here, we demonstrated a possible connection between the near-critical branching dynamics of the NALSM liquid and the edge-of-chaad transition (See Appendix A. The critical branching transition has been extensively used to model critical dynamics in brain networks (27] [6).. Focusing on the computational benefits of criticality, machine learning has mostly examined network dynamics at the edge-of-chaos transition. Although the presence of one transition does not guarantee the existence of the other, both transitions are well connected to the same result, an improved computational performance (25). Indeed, the computational performance of systems poised at a critical phase transition has been widely studied both experimentally [22] and theoretically [23], and are well-
connected to both edge-of-chaos [19] 20 and critical branching transitions [25] 15 Networks
operating at near-criticality are believed to have simultaneous access to the computational properties (learning and memory) of both phases, which results in 1) maximizing their information processing capacity [22). 2) optimizing their dynamical range [93] 24]. and 3) expanding their number of metastable states [25]. Hence, it is not surprising that the NALSM's astrocyte imposed near-critical branching dynamics resulted in improved accuracy and generalization capabilities as observed in LSMSs with edge-of-chanl dynamics [19 20), while adding the benefit of a neuromorphic compatibility and self organized criticality.
Our work shows how insights from modern cellular neuroscience can synergize with neuromorphic computing, and lead to novel intelligent systems, spurring the dialogue between artificial intelligence and brain sciences. Indeed, given that the known neuronal mechanisms are too slow and uncoordinated in the brain to modulate STDP [3 94 32). it is an open question how neurons modulate synaptic plasticity. Our demonstration that the distinct temporal and spatial mechanisms of astrocytes may modulate STDP and subsequently regulate network dynamics, questions the neuron as the only processing unit in the brain 995-97| In that sense, it helps in dismantling the 100-year old dogma that "brain = neurons", and tackle the absence of astrocytes in both prevailing computational hypotheses on how the brain learns and efforts to translate such knowledge to effective models of intelligence.
By showing how astrocyte-modulate STDP can maximize computational performance near criticality, we aimed to broaden the applicability of the LSM to complex spatio-temporal problems that require integration of data over multiple sources and time-scales, thereby, making LSMs suitable for real-life applications of edge computing. Our so far results suggest that this is a direction worth pursuing.
10

---

Acknowledgements
This work is supported by the National Center for Medical Rehabilitation Research (NIH/NICHI) 112HD099422 Grant and by the Rutgers Office of Research and Innovation. Any findings, conclu- sions, and opinions expressed in this material are those of the authors and do not necessarily reflect the views of the NIH or Rutgers University.
References
[1] Jack D. Kendall and Suhas Kumar. The building blocks of brain-inspired computer. Applied Physics Reviews, 7(1):01 1305, 2020. doi: 10.3063/1.5 155////..29366....
[2] Guangzhi Tang, Neelesh Kumar, and Konstantinos Michmizos. Reinforcement co-learning of deep and spiking neural networks for energy-efficient mapless navigation with neuromorphic hardware. 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 6090-6097, 2020. doi: 10.1 1.11100//004554..2000 2020.999999.99444.
[3] Mike Davies, Narayan Srinivasa, Tsung-Han Han Lin, Gautham Chinya, Yongqiang Cao, Sri Harsha Choday, Georgios Dimou, Prasad Joshi, Nabil Imam, Shweta Jain, Yuyun Liao, Chit-Kwan Lin, Andrew Lines, Ruokun Liu, Deepak Mathaikutty, Steven McCoy, Arnab Paul, Jonathan Tse, Guruguhanathan Venkataramanan, Yi-Hsin Weng, Andreas Wild, Yoonseok Yang, and Hong Wang. Loihi: A neuromorphic manycore processor with on-chip learning. IEEE Micro, 38(1): 82-99, 2018. doi: 10.1 10.110//MM.000 2018. 10018.1122 112130359,
[4] Guangzhi Tang, Arpit Shah, and Konstantinos P. Michmizos. Spiking neural network on neu- romorphic hardware for energy-efficient unidimensional slam. 2019 IEEE/RSJ Interna- tional Conference Intelligent Robots and Systems (IROS), pages 4176-4181, 2019, doi: 10.1 101100900440997.2011 988668644
[5] Keyan Cao, Yefan Liu, Gongjie Meng, and Qimeng Sun. An overview edge computing research. IEEE Access, 8:85714-85728, 2020. doi: 10.1 1091AACCSSSSS 200.02997220299977.
[6] Jiasi Chen and Xukan Ran. Deep learning with edge computing: A review. Proceedings of the IEEE, 165551665511677 2019. doi: 10. 109/JPR0C.22199.22219999
[7] Wolfgang Maass, Thomas Natschläger, and Henry Markram. Real-time computing without stable states: A new framework for neural computation based perturbations. Neural Computation, 14(11):2531-256) 2002. doi: 10. 122118899966077774777955..
[8] Kudithipudi Dhireesha Soures Nicholas. Deep liquid state machines with neural plasticity for video activity recognition. Frontiers in Neuroscience, 13:686, 2019. doi: 10.3389/minss2019 00686.
[9] Wachirawit Ponghiran, Gopalakrishnan Srinivasan, and Kaushik Roy. Reinforcement learning with low-complcxity liquid state machines. Frontiers in Neuroscience, 13:883, 2019. ISSN 1662-453X. doi: 10.3389/fnins.2011
[10] Qian Wang and Peng Li. D-Ism: Deep liquid state machine with unsupervised recurrent reservoir mning. In 20/6 23rd International Conference on Patlern Recognition (ICPR), pages 2652-2657, 2016. doi: 10. 10.1109/CPR.2016..999 79000355.
[1I] Shiya Liu, Lingjia Liu, and Yang Yi. Quantized reservoir computing on edge devices for communication applications. In 2020 IEEE/ACM Symposium on Edge Computing (SEC), pages 445-449, 2020. doi: 10.
[12] Shiming Li, Lei Wang, Shiying Wang, and Weixia Xu. Liquid state machine applications map- ping for noc-based neuromorphic platforms. In Dezun Dong, Xiaoli Gong, Cunlu Li, Dongsheng Li, and Junjie Wu, editors, Advanced Computer Architecture, pages 277-289, Singapore, 2020. Springer Singapore, ISBN 978-981-15-8113-9.
[13] Bon Woong Ku, Yu Liu, Yingyezhe Jin, Sandeep Samal, Peng Li, and Sung Kyu Lim. Design and architectural co-optimization of monolithic 3d liquid state machine-based neuromorphic processor. 2018 55th ACM/ESDA//EFF Design Automation Conference (DAC), pages 1-6, 2018. doi: 2.0.///AA..11.886655858

---

[14] Josep L. Rosselló, Miquel L. Alomar, Antoni Morro, Antoni Oliver, and Vincent Canals. High- density liquid-state machine circuitry for time-series forecasting. International Journal of Neural Systems, 26(05):1550033 2016. doi: 10.
PMID: 26906454.
[15] Ismael Balafrej and Jean Rouat. P-critical: reservoir autoregulation plasticity rule for neuromorphic hardware, 2020.
[16] Simon Brodeur and Jean Rouat. Regulation toward self-organized criticality in a recurrent spiking neural reservoir. In Alessandro E. P. Villa, Wlodzislaw Duch, Péter Érdi, Francesco Masulli, and Günther Palm, editors, Artificial Neural Networks and Machine Learning ICANN 2012, pages 547-554, Berlin, Heidelberg, 2012. Springer Berlin Heidelberg.
[17] Yingyezhe Jin and Peng Li. Ap-stdp: A novel self-organizing mechanism for efficient reservoir computing. In 2016 International Joint Conference on Neural Networks (IJCNN), pages 1158- 1165, 2016. doi: 10.1 W009UUCNN2201..77222
[18] D. Norton and D. Ventura. Preparing more effective liquid state machines using hebbian learning. In The 2006 IEEE International Joint Conference on Neural Network Proceedings, pages 4243-4248, 2006. doi: 10.1109/ICCN.2206.2499999
[19] Robert Legenstein and Wolfgang Maass. Edge of chaos and prediction of computational performance for neural circuit models. Neural Networks, 20(3):323 334, 2007. ISSN 0893- 6080. doi: h...016/..nuumtt20000 .0.0007..0077...
Echo State Networks and Liquid State Machines.
[20] Nils Bertschinger and Thomas Natschläger. Real-time computation at the edge of chaos in recurrent neural networks. Neural Computation, 16(7):1413-1436 2004. doi: 10.1162/
089976604333057453
[21] Chris G. Langton. Computation at the edge of chaos: phase transitions and emergent com- putation. Physica D: Nonlinear Phenomena, 42(1):12 37, 1990. ISSN 0167-2789. doi:
0L.0016016--278999990066-4
[22] Woodrow L. Shew, Hongdian Yang. Shan Yu, Rajarshi Roy, and Dietmar Plenz. Information capacity and transmission are maximized in balanced cortical networks with neuronal avalanches. Journal of Neuroscience, 31(1):55 63, 2011. ISSN 0270-6474. doi: 10. 10.1523/NEURROCC. 4637-10.2011.
[23] Lucilla de Arcangelis and Hans J. Herrmann. Learning as phenomenon occurring in critical state. Proceedings of the National Academy of Sciences, 107(9):3777-391, 2010. ISSN 0027-8424. doi: 10. 10.1073/pnas.00 0991990177.
[24] Osame Kinouchi and Mauro Copelli. Optimal dynamical range of excitable networks at criticality. Nature Physics, 2(5):348-33 2006. doi: 10. n0.10389/
[25] Clayton Haldeman and John M. Beggs. Critical branching captures activity in living neural networks and maximizes the number of metastable states. Phys. Rev. Lett., 94:058101, Feb 2005. doi: 10. www.PhyyyReeLet.9..5555...
[26] John M. Beggs and Dietmar Plenz. Neuronal avalanches in neocortical circuits. Journal of Neuroscience, 23635) 2003. ISSN 0270-6474. doi: 10.1523/IEURROCC. 23-35-11167. 2003.
[27] Oren Shriki, Jeff Alstott, Frederick Carver, Tom Holroyd, Richard N.A. Henson, Marie Smith, Richard Coppola, Edward Bullmore, and Dietmar Plenz. Neuronal avalanches in the resting meg of the human brain. Journal of Neuroscience, 33(16):7079-7090, 2013. ISSN 0270-6474, doi:
(utrrosceeeee33(11)7777----00000 10.1123/NNERROSC.. 12.2011.
[28] Dante R. Chialvo. Emergent complex neural dynamics. Nature Physics, 6:744-750, 2010. ISSN 1745-2481. doi: 10.1038/nphys1803.
[29] Daniel E. Feldman. Synaptic mechanisms for plasticity in neocortex. Annual Review of Neuro- science, 32(1):33-55, 2009. doi: 10. 1444/aaauuee..ccco0055555000
www.:nncuooo011050 135088..1116666 16. PMID: 19400721.
12

---

[30] Nigel Stepp, Dietmar Plenz, and Narayan Srinivasa. Synaptic plasticity enables adaptive self- tuning critical networks. PLOS Computaaionall Biology, 11(1):1-28, 01 2015. doi: 10.1371/ journal.pebi. 1004043.
[31] Friedemann Zenke and Wulfram Gerstner. Hebbian plasticity requires compensatory processes multiple timescales. Philosophical Transactions of the Royal Society B: Biological Sciences, 372(1715):20160) 2017. doi: 10. 10.1009//tb.2000 2016.02,
[32] Alanna Watt and Niraj Desai. Homeostatic plasticity and stdp: keeping a neuron's cool in fluctuating world. Frontiers in Synaptic Neuroscience, 2:5, 2010. ISSN 1663-3563. doi:
00.3389/fmsyy.211000000.
[33] L. F. Abbott and Sacha B. Nelson. Synaptic plasticity; taming the beast. Nature Neuroscience, 31)111778-1 2000, ISSN 1546-1726. doi: 10. 10.1038/81453.
[34] Gertrudis Perea and Alfonso Araque. Glia modulates synaptic transmission. Brain Research Reviews, 63(1):93-102, 2010. ISSN 0165-0173.
[35] Xiaoning Han, Michael Chen, Fushun Wang. Martha Windrem, Su Wang, Steven Shanz, Qiwu Xu, Nancy Ann Oberheim, Lane Bekar, and Sarah Betstadt. Forebrain engraftment by human glial progenitor cells enhances synaptic plasticity and learning adult mice. Cell Stem Cell, 12 (3):342-353, 2013. ISSN 1934-5909.
[36] Shivendra Tewari and Vladimir Parpura. A possible role of astrocytes in contextual memory retrieval: An analysis obtained using quantitative framework. Frontiers in Computational Neuroscience, 7:145, 2013. ISSN 1662-5188. doi: (..3399fnnomm2213..00
[37] Marta Navarrete and Alfonso Araque. Endocannabinoids potentiate synaptic transmission through stimulation of astrocytes. Neuron, 68(1): 2010. ISSN 0896-6273.
[38] Adar Adamsky, Adi Kol, Tirzah Kreisel, Adi Doron, Nofar Ozeri- Ozgri-Engelhard,, Talia Melcer, Ron Refaeli, Henrike Horn, Limor Regev, and Maya Groysman. Astrocytic activation generates de novo neuronal potentiation and memory enhancement. Cell, 174(1):59-71, e14, 2018. ISSN 0092-8674.
[39] Francesco Petrelli, Glenn Dallérac, Luca Pucci, Corrado Cali, Tamara Zehnder, Sébastien Sultan, Salvatore Lecea, Andrea Chicea, Andrei Ivanov, Cédric S. Asensio, Vidar Gundersen, Nicolas Toni, Graham William Knott, Fulvio Magara, Jürg Gertsch, Frank Kirchhoff, Nicole Déglon, Bruno Giros, Robert H. Edwards, Jean-Pierre Mothet, and Paola Bezzi. Dysfunction of homeo- static control of dopamine by astrocytes in the developing prefrontal cortex leads to cognitive impairments. Molecular Psychiatry, 25(4):732-749, 2020. doi: 10. 0000:/441100081--00000
[40] Tiina Manninen, Ausra Saudargiene, and Marja- Linne. Astrocyte-mediated spike-timing- dependent long-term depression modulates synaptic properties in the developing cortex. PLOS Computational Biology, (66(1)::1-20 11 2020. doi: (0.137//journal.pp pct......i..6.... 1008360.
[41] Jérémie Sibille, Ulrike Pannasch, and Nathalie Rouach. Astroglial potassium clearance con- tributes to short-term plasticity of synaptically evoked currents at the tripartite synapse. The Journal of Physiology, 592(1):87-102, 2014. doi: 10.1
[42] Rogier Min and Thomas Nevian. Astrocyte signaling controls spike sion at neocortical synapses. Nature Neuroscience, 15(5):746-75,, 2012. doi: 10.1038/nn.3075
depres-
[43] Alexander Stanley Thrane, Vinita Rangroo Thrane, Douglas Zeppenfeld, Nanhong Lou, Qiwu Xu, Erlend Arnulf Nagelhus, and Maiken Nedergaard. General anesthesia selectively disrupts astrocyte calcium signaling in the awake mouse cortex. Proceedings of the National Academy of Sciences, 109(4)):187441899) 2012. ISSN 0027-8424. doi: 10. 1atis.12944999 109.
[44] Jeannine Foley, Tamara Blutstein, Younn Lee, Christophe Erneux, Michael M. Halassa, and Philip Haydon. Astrocytic ip3/ca2+ signaling modulates theta rhythm and rem sleep. Frontiers in Neural Circuits, 11:3, 2017. ISSN 1662-51 10. doi: 2..338//mcir.2217700000
13

---

[45] Laura Bojarskaite, Daniel M. Bjørnstad, Klas H. Pettersen. Céline Cunen, Gudmund Horn Hermansen, Knut Sindre Abjørsbräten, Rolf Sprengel, Koen Vervacke, Wannan Tang, Rune Enger, and Erlend A. Nagelhus. Astrocytic ca2+ signaling is reduced during sleep and is involved in the regulation of slow wave sleep. Nature Communications, 11:3240, 2020. ISSN 2041-1723.
doi: 10. 1001///41477-----
[46] Ashley M. Ingiosi, Christopher Hayworth, Daniel O. Harvey, Kristan G. Singletary, Michael J. Rempe, Jonathan P. Wisor, and Marcos G. Frank. A role for astroglial calcium in mammalian sleep and sleep regulation. Current Biology, 30:4373-4383.c7, 2020. doi: 10.1016/f.cub.20000 wi1.2.00.00000 08.052.
[47] Enzo Tagliazucchi, Dante R. Chialvo, Michael Siniatchkin, Enrico Amico, Jean-Francois Brichant, Vincent Bonhomme, Quentin Noirhomme, Helmut Laufs, and Steven Laureys. Large- scale signatures of unconsciousness are consistent with a departure from critical dynamics. Journal of The Royal Society Interface, 13(114)20011007,, 2016. doi: 20.109 1027.
[48] Timothy Bellay, Andreas Klaus, Saurav Seshadri, and Dietmar Plenz. Irregular spiking of pyramidal neurons organizes as scale-invariant neuronal avalanches in the awake state. el.ife, 4: e07224, jul 2015. ISSN 2050-084X. doi: (0.7554/eL.ffe002222
[49] Gerald Hahn, Adrian Ponce-Alvarez, Cyril Monier, Giacomo Benvenuti, Arvind Kumar, Frédéric Chavane, Gustavo Deco, and Yves Frégnac. Spontancous cortical activity is tran- siently poised close to criticality. PLOS Computational Biology, 13(5):1-29, 05 2017. doi:
10.1371/jourmal.pchii 1005543.
[50] Viola Priesemann, Mario Valderrama, Michael Wibral, and Michel Le Van Quyen. Neuronal avalanches differ from wakefulness to deep sleep evidence from intracranial depth recordings in humans. PLOS Computational Biology, 9(3): 03 2013. doi: 10.137 //jouraalll.pbb 1002985.
[51] Erik D. Fagerholm, Romy Lorenz, Gregory Scott, Martin Dinov, Peter J. Hellyer, Nazanin Mirzaei, Clare Leeson, David W. Carmichael, David J. Sharp, Woodrow L. Shew, and Robert Leech. Cascades and cognitive state: focused attention incurs subcritical dynamics. Journal of Neuroscience, 35(11):4626 2015. ISSN 0270-6474. doi: 10. N0.122//NEURSSCI3664.. 14. 2015.
[52] Vladimir Parpura and Philip G. Haydon. Physiological astrocytic calcium levels stimulate glutamate release modulate adjacent neurons. Proceedings of the National Academy of Sciences, (7(15):8629-866 2000. ISSN 0027-8424. doi: 19.1073/ppas.97.15.66...
[53] Catherine A. Christian and John R. Huguenard. Astrocytes potentiate gabaergic transmission in the thalamic reticular nucleus via endozepine signaling. Proceedings of the National Academy of Sciences, 10(55)) 20283, 2013. doi: 10. 10.3033/pnas. 13 80011100
[54] Sara Mederos, Candela González-Arias and Gertrudis Perea. Astrocyte- neuron networks: multilane highway of signaling for homeostatic brain function. Frontiers in Synaptic Neuro- science, 10(45). 2018. ISSN 1663-3563, doi: 10.3389/fnsyn22118.0000
[55] Biji Shigetomi, Sandip Patel, and Baljit S. Khakh. Probing the complexities of astrocyte calcium signaling. Trends in Cell Biology, 26(4):300-312, 2016. ISSN 0962-8924.
[56] Gertrudis Perea, Mriganka Sur, and Alfonso Araque Neuron-glia networks: integral gear of brain function. Frontiers in Cellular Neuroscience, 8(378), 2014. ISSN 1662-5102, doi:
10. (0.33899/nneel221440007
[57] Alfonso Araque, Giorgio Carmignoto, Philip G Haydon, Stéphane HR Oliet, Richard Robitaille, and Andrea Volterra. Gliotransmitters travel in time and space. Neuron, 81(4):728-733 2014. ISSN 0896-6273.
[58] Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner. Gradient- learning applied document recognition. Proceedings of the IEEE, 86(11):228-22324 1998. doi: 10. 12975.77679.
[59] Garrick Orchard, Ajinkya Jayawant, Gregory K. Cohen, and Nitish Thakor. Converting static image datasets to spiking neuromorphic datasets using saccades. Frontiers in Neuroscience, 437, 2015. ISSN 1662-453X. doi: 13389//inns.21155004377.
14

---

[60] Han Xiao. Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms, 2017.
[61] Abigail Morrison, Markus Diesmann, and Wulfram Gerstner. Phenomenological models of synaptic plasticity based on spike timing. Biological Cybernetics, 98(6):459 478, 2008. ISSN 1432-0770. doi: 10. 07777/0004220000-0222
[62] Wenrui Zhang and Peng Li. cnermation-theoretee intrinsic plasticity for online unsupervised learning in spiking neural networks. Frontiers in Neuroscience, 13:31, 2019, ISSN 1662-453X.
doi: 10.3389/fmins.20) 19.00031.
[63] Wolfgang Maass, Robert Legenstein, and Nils Bertschinger. Methods for estimating the computational power and generalization capability of neural microcircuits. Proceedings of the 17th International Conference on Neural Information Processing Systems, NIPS 045, page 865-872, Cambridge, MA, USA, 2004. MIT Press.
[64] Parami Wijesinghe, Gopalakrishnan Srinivasan, Priyadarshini Panda, and Kaushik Roy. Analysis of liquid ensembles for enhancing the performance and accuracy of liquid state machines. Frontiers in Neuroscience, 13:504, 2019. ISSN 1662-453X. doi: 20338//fnins.201900055175
[65] Joschka Boedecker, Oliver Obst, Joseph T Lizier, N Michael Mayer, and Minoru Asada. Information processing in echo state networks at the edge of chaos. Theory Biosci., 131(3): 205-13, 2012. doi:
[66] Andre S. Ribeiro, Stuart A. Kauffman, Jason Lloyd-Price, Bjöm Samuelsson, and Joshua E. S. Socolar. Mutual information in random boolean models of regulatory networks. Phys. Rev. E, 77: 01 1901, Jan 2008. doi: 10. 10.1100/PyysRee.77.011990
[67] Lukasz Kuśmierz, Shun Ogawa, and Taro Toyoizumi. Edge of chaos and avalanches in neural networks with heavy-tailed synaptic weight distribution. Phys. Rev. Lett., 125:028101, Jul 2020. doi: 10. 103/PhysReVLet. 125.022101
[68] C. van Vreeswijk and H. Sompolinsky. Chaos in neuronal networks with balanced excitatory and inhibitory activity. Science, 274(5293); 1996. doi: 10. 10.11266scceme:224555377.222
[69] Srdjan Ostojic. Two types of asynchronous activity in networks of excitatory and inhibitory spiking neurons. Nature Neuroscience, 17:594 600, 2014. ISSN 1546- 1726. doi: 10.1038/nn. 3658.
[70] Kanaka Rajan, L. Abbott, and Haim Sompolinsky. Stimulus-dependent suppression of chaos in recurrent neural networks. Phys. Rev. E, 82:01 1903, Jul 2010. doi: 10. 10.1103/PhysRevE.822 01 1903.
[71] Bin Zhou, Yun -ia Zuo, and Ruo-Tian Jiang. Astrocyte morphology: diversity, plasticity, and role in neurological diseases. CNS Neuroscience Therapeutics, 25(6):665-673, 2019. doi: 10.1 11 //////ss.11....
[72] Dongcheng Zhao, Yi Zeng, Tielin Zhang, Mengting Shi, and Fei fei Zhao. Glsnn: multi-layer spiking neural network based on global feedback alignment and local stdp plasticity. Frontiers in Computational Neuroscience, 14, 2020.
[73] Arash Samadi, T. Lillicrap, and D. Tweed. Deep learning with dynamic spiking neurons and fixed feedback weights. Neural Computation, 29:578-602. 2017.
[74] Tielin Zhang, Yi Zeng, Dongcheng Zhao, and Bo Xu. Brain-inspired balanced tuning for spiking neural networks. In Proceedings of the Seenty-Sevnnth International Joint Conference on Artificial Intelligence, IJCAI-18, pages 1653-1659, International Joint Conferences Artificial Intelligence Organization, 7 2018. doi: 10. 10.24963/jcca.201133 2018//29.
[75] Peter Diehl and Matthew Cook. Unsupervised learning of digit recognition using spike-timing- dependent plasticity. Frontiers in Computational Neuroscience, 9:99, 2015. ISSN 1662-5188. doi: 33389//ncom.2215000000
15

---

[76] Hesham Mostafa. Supervised learning based on temporal coding in spiking neural networks. IEEE Transactions on Neural Networks and Learning Systems, 29:3227-3235, 2018.
[77] Maryam Mirsadeghi, Majid Shalchian, Saeed Reza Kheradpisheh, and Timothée Masquelier. Stidi-bp: Spike time displacement based error backpropagation in multilayer spiking neural networks. Neurocomputing, 427:131-140, 2021. ISSN 0925-2312. doi: http:://doi.org/1.111111 neucom.2020.1 1.002..
[78] Peter O'Connor and M. Welling, Deep spiking networks. ArXiv, abs/1602.08333 2016.
[79] Jun Haeng Lee, Tobi Delbruck, and Michael Pfeiffer. Training deep spiking neural networks using backpropagation. Frontiers in Neuroscience, 10:508, 2016. ISSN 1662-453X. doi: 00.3389/finns.2166.000
[80] Yujie Wu, Lei Deng, Guoqi Li, Jun Zhu, and Luping Shi. Spatio-temporal backpropagation for training high-performance spiking neural networks. Frontiers in Neuroscience, 12:331, 2018. ISSN 1662-453X.
doi: 10.3389/fnins.200 2018.0001333333
[81] Jacques Kaiser, Hesham Mostafa, and Emre Neftci. Synaptic plasticity dynamics for deep continuous local learning (decolle). Frontiers in Neuroscience, 14:424, 2020. ISSN 1662-453X. doi: ..338///iinss222200044444
[82] Qianhui Liu, Haibo Ruan, Dong Xing, H. Tang, and Gang Pan. Effective aer object classification using segmented probabilit-maximizztie maximization learning in spiking neural networks. In AAAI, 2020.
[83] S. Shrestha and G. Orchard. Slayer: Spike layer error reassignment in time. In NeurIPS, 2018.
[84] Tielin Zhang, Yi Zeng, Dongcheng Zhao, and Mengting Shi. A plasticity- centric approach to train the non-differential spiking neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, 2018.
[85] Yunzhe Hao, Xuhui Huang, Meng Dong, and Bo Xu. A biologically plausible supervised learning method for spiking neural networks using the symmetric stdp rule. Neural Networks, 121:387-395, 2020. ISSN 0893-6080. doi: https://doi.org/// /00.
[86] Saeed Reza Kheradpisheh, Maryam Mirsadeghi, and Timothée Masquelier. BS4NN: binarized spiking neural networks with temporal coding and learning. CoRR, abs/2007.04033 2020.
[87] Han Ju, Jian-Xin Xu, Edmund Chong, and Antonius M.J. VanDongen. Effects of synaptic connectivity on liquid state machine performance. Neural Networks, 38:39-51, 2013. ISSN 0893-6080. doi: https://doi.org org.//....r/....... 10. 1016/j newnet. 1..003.
[88] Filip Ponulak and Andrzej Kasiński. Supervised learning in spiking neural networks with resume: Sequence learning, classification, and spike shifting. Neural Comput., 22(2):467-500, February 2010. ISSN 0899-7667.
doi: 10. 10.1162/neco.2000 1--0-901-
[89] Robert Urbanczik and Walter Senn. A Gradient Learning Rule for the Tempotron. Neural Computation, 21(2):30-352, 02 2009. ISSN 0899-7667. doi: 10.
[90] Robert Gütig and Haim Sompolinsky. The tempotron: a neuron that learns spike timing-based decisions. Nature Neuroscience, 9(3):220-422 2006. doi: 10.1038/mn1643.
[91] Guangzhi Tang, Ioannis E. Polykretis, Vladimir A. Ivanov, Arpit Shah, and Konstantinos P. Michmizos. Introducing astrocytes on ncuromorphic processor: Synchronization, local plasticity and edge of chaos. ACM Proceedings of 2019 Neuroinspired Computing Elements (NICE 2019),
1(1):1-10, 2019. doi: arXiv: 1et.11997602000
[92] Alex Roxin, Nicolas Brunel, David Hansel, Gianluigi Mongillo, and Carl van Vreeswijk. On the distribution of firing rates in networks of cortical neurons. Journal of Neuroscience, 31(45): 16217-16226, 101 ISSN 0270-6474. doi: 10.152/INEUROSCL.1677 1.2011.
[93] Woodrow L. Shew, Hongdian Yang, Thomas Petermann, Rajarshi Roy, and Dietmar Plenz. Neuronal avalanches imply maximum dynamic range in cortical networks at criticality. Journal of Neuroscience, 19549): 2009. ISSN 0270-6474. doi: 10. 12//NNNUNUOSSC
18864-99.200 09.2009.
16

---

[94] Friedemann Zenke, Guillaume Hennequin, and Wulfram Gerstner. Synaptic plasticity in neural networks needs homeostasis with fast rate detector. PLOS Computational Biology, 9(11):1-14, 11 2013. doi: 10. /joo///uuraaa 103...0003330.
[95] Michael M. Halassa and Philip G. Haydon. Integrated brain circuits: astrocytic networks modulate neuronal activity and behavior. Annual Review of Physiology, 72(1):335-355, 2010.
doi: 10. 146/annurev- 4 1399--133399
[96] Eduardo E. Benarroch. Neuron-astrocyte interactions: partnership for normal function and disease in the central nervous system. Mayo Clinic Proceedings, 80(10):1326-1338 2005. ISSN 0025-6196.
[97] Dionysia T. Theodosis, Dominique A. Poulain, and Stéphane H. R. Oliet. detivity structural and functional plasticity of astrocyte-neuron interactions. Physiological Reviews, 88 (3):983-1000 2008. doi: 10.11522/byyreex00006.20...
17


Reviews:
[{"review_id": "c6sVrOj7cQP", "reviewer": "Reviewer_rDJf", "summary": "This paper extends liquid state machine (LSM) with a biologically inspired astrocyte model (NALSM) to optimize the neuronal dynamics in LSM at the edge-of-chaos, to improve the accuracy and stability of LSMs. With NALSM, the spiking model can maintain a high accuracy without re-tuning parameters for different datasets. Experiments with MNIST and N-MNIST dataset demonstrate that NALSM outperforms classic LSM and other STDP-based LSM approaches, and most existing spiking neuron models including multi-layer SNNs.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "1. Overall, the idea of borrowing astrocyte mechanisms to build a self-organized dynamic controller for LSM is novel and interesting. The paper is well organized and presented. Although LSM model is currently a narrow topic, the way of thinking is inspiring and can be beneficial to other spiking neuron models. \n\n2. Why the spike-timing-dependent plasticity (STDP) based inter-connection learning is beneficial for LSM models?\n\n3. Discussions and comparisons with other edge-of-chaos modulation approaches can be added to better demonstrate the advantages of NALSM.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes.", "review_text": "", "extra_scores": {"ethical_concerns": "No.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "AW6FNzDOfrx", "reviewer": "Reviewer_r3sw", "summary": "The manuscript proposes to improve a liquid state machine (implemented in spiking neurons) by adding astrocyte-inspried STDP-mechanism to automatically poise at a 'critical' state, as quantified by a branching factor. The submission claims that close to the so-called 'edge-of-chaos', the proposed networks have improved performance, as quantified in MNIST and N-MNIST.", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality:\n* liquid state machines (LSM) are a classical alternative to BPTT. Here are novel neuron-astrocyte LSM is being proposed and benchmarked, which seems to be original.\n\nQuality:\n* The presented study is a purely numerical work, a mathematical understanding of the neuron-astrocyte LSM is missing.\n* The paper claims that the NASLM self-regulate towards the edge of chaos, but this claim is not being supported by standard measures of chaos (e.g. Lyapunov exponents). Instead, a \"branching factor\" is being computed, which is being interpreted as a proxy of chaos. This claim is not supported by any further evidence.\n\n* Given the numerical nature of the study, important information is missing, e.g. how the baseline was optimized. Moreover, a comparison not only in terms of performance but also in terms of total computational cost (in flops or GPU hours) is missing.\n\nClarity: The manuscript is written clearly and very understandable. \n\n* significance:\nThe significance of the presented work seems to be limited, as the NASLM is only benchmarked against other versions of SLM, so it is not clear how it compares to more widely used RNN training methods.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The presented does not systematically vary various parameters of the proposed NASLM to investigate its impact on both chaoticity and performance so the overall claim that the performance of liquid state machines is improved closed to the edge of chaos using the proposed astrocyte-mediated mechanisms seems not fully supported.\nTo spell it out in more detail:\n* Quantification of chaos (Lyapunov exponent) is missing\n* Mathematical theory describing the self-tuning towards an 'edge-of-chaos' is missing\n* Mathematical theory linking 'edge of chaos' to superior performance is also missing.", "review_text": "", "extra_scores": {"ethical_concerns": "no", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5h"}, "extra_text": {}}, {"review_id": "0gIyArVySvf", "reviewer": "Reviewer_qHbj", "summary": "The authors propose of a way modulating the connections in a liquid state machine (LSM) so that it moves towards the edge of chaos (EOC) to maximize its performance for various tasks. They do this using by introducing an additional \"astrocyte\" that modulates the parameters of STDP in the neuron model -- specifically the depression term. The authors compare their model to other forms of LSMs.\n", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Update**: The authors response addressed most of my major concerns, and therefore I would like to upgrade my rating.\n\n## Originality: \nModulating the STDP parameters of a network in this particular way to move the network towards the edge of chaos is quite novel to my knowledge. The authors do an adequate discussion of related work. Being able to move the network towards EOC in an automated way using such a relatively simple mechanism is also a very nice feature to have for training liquids.\n\n## Quality: \nThe approach is well motivated and simple, and the empirical analysis is reasonable.  The authors compare their approach with a wide variety of other published architectures.\n\nTesting the model on further standard benchmarks for e.g. with TIMIT or other speech recognition tasks can make the performance results more convincing.\n\nThere are also differences in how STDP is applied between LSM+STDP and NALSM -- in NALSM the STDP continues to adjust weights when collecting spike count samples but not in the other case. Why?\n\nA bit more generally, what's the spiking rate of the liquid? This is a useful metric to see how efficient it could be on neuromorphic hardware and how relatable it is to biology.\n\nA comparison to backpropagation based approaches for setting liquid weights such as [1] could also be made:\n\n[1] Subramoney, Anand, Franz Scherr, and Wolfgang Maass. \"Reservoirs learn to learn.\" arXiv preprint arXiv:1909.07486 (2019).\n\nMinor: how important is it to have distance dependent connectivity?\n\n\n## Clarity: \nWhile the text does convey the core points well, the writing and structure leave much to be desired. There are various redundant parts and some details are missing in the text. Some examples:\n* eqn. 3: C is not defined.\n* l.105: What does $ A_{-}^{astro} \\rightarrow A_{-} $ mean?\n* para starting at l.214 is very redundant with figure caption. Ditto para starting l.285\n\n\nOther comments:\n* l.130: The sentence reads as if all the weights in the liquid are equal.\n* I would suggest describing LSM+AP-STDP at least briefly.\n* Ditto for how the branching factor is calculated.\n* It should be explicitly mentioned that the results in Fig. 2 are without astrocytes.\n* Need more details on the kernel quality metric.\n\nMinor:\n* In many places, references are made using just the number, which could be ambiguous. For e.g. 158: 2.1 should be Sec. 2.1, l.164 etc.\n\n## Significance: \nBeing able to automatically move a liquid towards EOC can be quite important to avoid manual tuning often required for liquids. But the authors use a heuristic approach to doing this, which might limit how much it can be built on. But given the empirical results, this method can be extremely useful for practitioners.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors do not discuss limitations of their work.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}, {"review_id": "02ySKmvjutH", "reviewer": "Reviewer_879c", "summary": "Liquid State Machines (LSM)  circumvents the need for backpropagation and allows the training of recurrent neural networks by local plasticity in the readout weights. Previous studies have shown that optimal performance is achieved near a critical phase transition of the reservoir (the liquid). However, the exact parameters of the critical phase transition in the liquid vary and depend on the input. In this work, the authors present a biologically inspired model that allows self-organization near the critical transition. Here, the authors propose a biologically inspired model in which the liquid self-organizes near the critical point. The model is composed of a reservoir of integrate-and-fire neurons. The synaptic efficacies within have unsupervised spike-time dependent plasticity (STDP). The authors propose an additional dynamical variable akin to the activity of astrocytes in the circuit, which modifies the STDP learning rates. The model self-organizes close to the critical point. The authors train several networks on the MNIST and N-MNIST dataset (a spiking version of MNIST, made for neuromorphic computing). They show that the SLM model with astrocyte-dependent STDP plasticity outperforms other LSM models.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "### Impact\n\nThe main novelty in this work is suggesting that astrocytes may be responsible for the self-organizing of cortical circuits close to criticality. This is an interesting concept and may pave the way for more works studying cortical circuits as liquid state machines.\n\n### Edge of chaos and criticality\n\nIt is not clear to me from the paper which critical transition, or edge-of-chaos criticality, the authors imply in the text.  \n\n(I) The authors refer to the critical point as the edge-of-chaos in the liquid dynamics (see, e.g., the caption of figure 1). I do not believe that the studied critical point is the onset of chaos of the liquid state, and there are several reasons for that:\n\n1. A spiking model was previously argued not to show a chaotic phase transition (Engelken et al). It was later shown that for spiking, the transition is smoothed to a continuous crossover and the critical effects are strongly diminished (Kadmon and Sompolinsky, PRX, 2015)\n2. The connectivity model in eq (3), in which neurons are mostly connected to their neighbors is not known to show chaotic behavior (as far as I know). \n3. Strong spatial variance is needed to induce chaos, but it is not clear what the parameter 'C'  is in eq (3). In general, for networks that respect Dale's law, in order to get chaotic behavior, there needs to be a strong balance between excitatory and inhibitory populations [Van Vreeswijk and Sompolinsky 1998]. Alternatively, recent work has studied chaotic dynamics in heavy-tailed distribution [Kuśmierz et al., PRL, 2020]. The authors show neither in their model.\n4. There is no evidence that the transition in question is the transition to chaotic activity. The authors use a form of a branching factor (BF) as the order parameter (more on that below). BF>1 leads to critical excitability, and not chaotic behavior.\n\n(II) The authors refer to [22-24] as evidence of edge-of-chaos computation in the brain.  However, these works consider a critical-branching phase transition which is associated with the Mean-Field Directed Percolation (MF-DP) universality class. This is not the transition to chaos (unless in specific settings when the two coincide [see Kuśmierz et al. 2019]. The critical branching-criticality is defined when the branching factor is (BF=1), as implied in this paper. However, it is defined when a single spike (in the liquid!) elicit one spike on average (in the liquid!). The branching factor defined here is different (see my comment below). \n\n(III) Finally the authors may be referring to the edge-of-chaos in feedforward networks, as studied in [Hayou et al. (2019); Lee et al. (2018); Schoenholz et al. (2017)] for rate networks. However, this is not a critical point of the liquid and is not associated with improved computation shown in [19-21]. Furthermore, the edge-of-chaos in the feedforward networks mentioned is defined by the spectrum of the Jacobian of the connectivity matrix, which is not studied in the current paper, so the relation to it is not clear if it exists.\n\n### The use of the branching factor\n\nThe order parameter used to measure the dynamical transition in the work is the branching factor (BF). In the proposed model, the dynamics of the astrocytes are driven by a proxy for this measure. There are a few problems I find in this formalism:\n\n1. The criticality in [22-24] is defined with a branching factor of 1 when each spike in the liquid elicits one spike on average within the liquid. This is a different point than the information propagation in feedforward networks [e.g., Hayou et al. (2019); Lee et al. (2018); Schoenholz et al. (2017)]\n2. Equations (7) and (8) are not normalized to the size of the input and liquid populations. In the experimental results, the authors change the fraction of liquid neurons that are connected to the astrocyte. Unless I misunderstood, this should change the measure of BFproxy and the astrocyte dynamics. More importantly, is not clear what is the meaning of BF=1 in the case when there is no normalization to the number of neurons sampled.\n\n\n### Novelty\nThe paper emphasizes two results: that computation near criticality is better and that astrocyte is a way to get self-organized criticality (SOC). The first was discussed extensively in the past [19-21]. As for SOC, there have been numerous papers suggesting different schemes [e.g., Girardi-Schappo et al, Physical Review Research 2020]. While the idea of using astrocytes as a way to get SOM is interesting, it is not conceptually new in terms of computation, and I feel that this work will fit better in a biologically oriented venue.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "yes", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers agree that the main contribution of this paper, i.e. that the hypothesis that astrocytes help networks to self-organize close to criticality, is novel. However, the concrete impact of this contribution is somewhat unclear. Is this a statement about the function of astrocytes in biological neural circuits? If so, the paper would ideally argue in greater detail that their astrocytes model is realistic, and furthermore that other spike-rate adaptation mechanisms do not have similar, beneficial effects. Conversely, as a contribution to algorithms on neuromorphic hardware for tackling practical problems, this work would benefit from additional comparisons to alternatives for introducing stabilizing negative feedback mechanisms. Therefore, the overall contribution of this paper is below acceptance.


Decision: Accept (Poster)