Increasing Liquid State Machine Performance with Edge-of-Chaos Dynamics Organized by Astrocyte-modulated Plasticity

1 Introduction . With the recent rise of neuromorphic [ 1–4 ] and edge computing [ 5 , 6 ] , the liquid state machine ( LSM ) learning framework [ 7 ] has become an attractive alternative [ 8–11 ] to deep neural networks owing to its compatibility with energy-efficient neuromorphic hardware [ 12–14 ] and inherently low training complexity . Originally proposed as a biologically plausible model of learning , LSMs avoid training via backpropagation by using a sparse , recurrent , spiking neural network ( liquid ) with fixed synaptic connection weights to project inputs into a high dimensional space from which a single neural layer can learn the correct outputs . Yet , these advantages over deep networks come at the expense of 1 ) sub-par accuracy and 2 ) extensive data-specific hand-tuning of liquid weights . Interestingly , these 1Code and data available at https : //github.com/combra-lab/NALSM 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . two limitations have been targeted by several studies that tackle one [ 15 , 16 ] or the other [ 17 , 18 ] , but not both . This has limited the widespread use of LSMs in real-world applications [ 8 ] . In that sense , there is an unmet need for a unified , brain-inspired approach that is directly applicable to the emerging neuromorphic and edge computing technologies , facilitating them to go mainstream . As a general heuristic , LSM accuracy is maximized when LSM dynamics are positioned at the edgeof-chaos [ 19–21 ] and specifically in the vicinity of a critical phase transition [ 22–25 ] that separates : 1 ) the sub-critical phase , where network activity decays , and 2 ) the super-critical ( chaotic ) phase , where network activity gets exponentially amplified . Strikingly , brain networks have also been found to operate near a critical phase transition [ 26–28 ] that is modeled as a branching process [ 25 , 26 ] . Current LSM tuning methods organize network dynamics at the critical branching factor by adding forward and backward communication channels on top of the liquid [ 15 , 16 ] . This , however , results in significant increases in training complexity and violates the LSM ’ s brain-inspired self-organization principles . For example , these methods lack local plasticity rules that are widely observed in the brain and considered a key component for both biological [ 29 ] and neuromorphic learning [ 3 , 2 , 4 ] . A particular local learning rule , spike-timing-dependent plasticity ( STDP ) , is known to improve LSM accuracy [ 17 , 18 ] . Yet , current methods of incorporating STDP into LSMs further exacerbate the limitations of data-specific hand-tuning as they require additional mechanisms to compensate for the STDP-imposed saturation of synaptic weights [ 17 , 30–33 ] . This signifies the scarcity of LSM tuning methods that are both computationally efficient and data-independent . A long-neglected non-neuronal cell in the brain , astrocytes , is now known to play key roles in modulating brain networks [ 34–39 ] , from modifying synaptic plasticity [ 40–42 ] to facilitating switching between cognitive states [ 43–46 ] that have been linked to a narrow spectrum of dynamics around the critical phase transition [ 47–51 ] . The mechanisms that astrocytes use to modulate neurons include the integration of the activity of thousands of synapses into a slow intracellular continuous signal that feeds back to neurons by affecting their synaptic plasticity [ 52–55 , 42 ] . The unique spatiotemporal attributes [ 56 , 57 ] identified in astrocytes align well with the brain ’ s remarkable ability to self-organize its massive and highly recursive networks near criticality . That is why astrocytes present a fascinating possibility of forming a unified feedback modulation mechanism required to improve baseline LSM accuracy while eliminating data-specific hand-tuning . Here , we propose the neuron-astrocyte liquid state machine ( NALSM ) , where a biologically inspired astrocyte model organized liquid dynamics near a critical phase transition , by modulating STDP . We show that NALSM combined the computational benefits of both STDP and critical branching dynamics by demonstrating its accuracy advantage compared to other LSM methods on two datasets : 1 ) MNIST [ 58 ] , and 2 ) N-MNIST [ 59 ] . We demonstrate that , similar to its biological counterpart that handles new and unstructured information with robustness and versatility , NALSM maintains the state-of-the-art LSM performance without re-tuning training parameters for each tested dataset . We also show that a NALSM with a large enough liquid can attain comparable accuracy to fully-connected multi-layer spiking neural networks trained via backpropagation on 1 ) MNIST [ 58 ] , 2 ) N-MNIST [ 59 ] , as well as 3 ) Fashion-MNIST [ 60 ] . Our results suggest that the under-performance and high training difficulty of current neuromorphic methods can be addressed by harvesting neuroscience knowledge and further translating biological principles to computational mechanisms . 2 Methods . 2.1 The neuron-astrocyte liquid state machine . To construct the NALSM , we started with a baseline LSM model consisting of 2 layers : 1 ) a spiking liquid , and 2 ) a linear output layer . Next , we added STDP to the LSM liquid , forming the LSM+STDP model . We developed a biologically faithful leaky-integrate-and-modulate ( LIM ) astrocyte model , which we embedded in the LSM+STDP liquid , to form the NALSM . The process is formalized below . LSM Model We implemented the baseline LSM as a 3-dimensional neural network ( liquid ) consisting of 1 , 000 neurons surrounded by 1-dimensional layers of input and output neurons . Number of input neurons was 784 and 2 , 312 for MNIST and N-MNIST , respectively ( See Appendix A.1 ) . We used the leaky-integrate-and-fire ( LIF ) model [ 3 ] for input and liquid neurons , modeled as : dvi dt = − 1 τv vi ( t ) + ui ( t ) − θiσi ( t ) ( 1 ) ui ( t ) = ∑ j 6=i wij ( αu ∗ σj ) ( t ) + bi ( 2 ) where vi is the membrane potential and ui is the synaptic response current of neuron i , θi is the membrane potential threshold , σi ( t ) = ∑ k δ ( t − tki ) is the spike train of neuron i with tki being the time of the k-th spike , wij is the weight connecting neuron j to i , bi is the bias of neuron i , and αu ( t ) = τ−1u exp ( −t/τu ) H ( t ) is the synaptic filter with H ( t ) being the unit step function ( See Appendix A.2 ) . All LIF neurons had a 2 ms absolute refractory period . Liquid neurons were excitatory and inhibitory with 80 % /20 % ratio . Input neurons did not have an excitatory/inhibitory distinction and had random excitatory and inhibitory connections to liquid neurons . From here on , we will refer to connections between input neurons and liquid neurons as IL connections , inter-liquid connections as LL , and liquid to output connections as LO . In line with [ 7 ] , we created LL connections using probabilities based on Euclidean distance , D ( i , j ) , between any two neurons i , j : P ( i , j ) = C · exp ( − ( D ( i , j ) λ ) 2 ) ( 3 ) with closer neurons having higher connection probability . Parameters C and λ set the amplitude and horizontal shift , respectively , of the probability distribution ( See Appendix A.3 ) . Density of IL connections was 15 % . The output layer was a dense layer consisting of 10 linear neurons . LSM+STDP Model We added unsupervised , local learning to the LSM model by letting STDP change each LL and IL connection [ 61 ] , modeled as : dw dt = A+Tpre ∑ o δ ( t− topost ) −A−Tpost ∑ i δ ( t− tipre ) ( 4 ) where A+ = A− = 0.15 are the potentiation/depression learning rates and Tpre/Tpost are the pre/post-synaptic trace variables , modeled as , τ∗+ dTpre dt = −Tpre + a+ ∑ i δ ( t− tipre ) ( 5 ) τ∗− dTpost dt = −Tpost + a− ∑ o δ ( t− topost ) ( 6 ) where a+ = a− = 0.1 are the discrete contributions of each spike to the trace variable , τ∗+ = τ ∗ − = 10 ms are the decay time constants , tipre and t o post are the times of the pre-synaptic and post-synaptic spikes , respectively . We constrained connection weights to : 1 ) IL : [ −3 , 3 ] , 2 ) excitatory LL : [ 0 , 3 ] , and 3 ) inhibitory LL : [ −3 , 0 ] . We used the same STDP parameters for all models and experiments . LIM Astrocyte Model We developed the astrocyte model as a leaky integrator with a continuous output value Aastro− , expressed as : τasto dAastro− dt = −Aastro− + wastro ∑ i∈Nliq δ ( t− ti ) − wastro ∑ j∈Ninp δ ( t− tj ) + bastro ( 7 ) where Aastro− directly mapped to A− in equation ( 4 ) , bastro = A+ adjusted the astrocyte output to the fixed STDP potentiation learning rate , Nliq and Ninp are the sets of liquid and input neurons , respectively , andwastro set astrocyte responsiveness to network activity ( See Appendix A.4 ) . Ignoring the decay and bias terms , the astrocyte model computed the difference in the number of spikes produced by liquid neurons and input neurons . Functionally , this is equivalent to computing the ratio of spikes emitted by the liquid over the input neurons : BFproxy ( t ) = ∑ i∈Nliq δ ( t− ti ) ∑ j∈Ninp δ ( t− tj ) ( 8 ) Specifically , when the liquid produced more spikes than the input neurons , their difference was positive which translated to BFproxy > 1.0 , and vice versa . This approach to measure liquid dynamics acted as a network level approximation of the branching factor , σBF , which is normally evaluated for each neuron ( See 2.3.1 ) . We empirically confirmed that BFproxy = 1.0 aligned with the critical branching factor , σBF = 1.0 ( Fig . 1 B ) . Hence , as dynamics became progressively supercritical ( σBF > 1.0 ) , BFproxy became greater than 1 , which caused the LIM astrocyte to increase STDP depression learning rate above the fixed STDP potentiation learning rate ( Aastro− → A− > A+ ) . This caused STDP to decrease the average weight of LL and IL connections , which decreased number of spikes produced by the liquid and made dynamics less super-critical ( Fig . 1 B ) . The reverse occurred as dynamics became progressively sub-critical . As a result of astrocyte modulation , liquid dynamics oscillated between sub-critical and super-critical until eventual stabilization near the critical branching factor ( See Appendix A.4 ) . NALSM Model We completed NALSM by adding the LIM astrocyte to the LSM+STDP model ’ s liquid ( Fig . 1 A ) . As described above , the LIM astrocyte integrated activity from input and liquid neurons , and continuously controlled the STDP depression learning rate . 2.2 Training . Model training was done in 3 steps : 1 ) initialization of IL and LL liquid connections , 2 ) passing all data through the liquid resulting in liquid neuron spike counts , and 3 ) training the output layer on the spike counts . The steps are further detailed below .

Summary: Liquid State Machines (LSM)  circumvents the need for backpropagation and allows the training of recurrent neural networks by local plasticity in the readout weights. Previous studies have shown that optimal performance is achieved near a critical phase transition of the reservoir (the liquid). However, the exact parameters of the critical phase transition in the liquid vary and depend on the input. In this work, the authors present a biologically inspired model that allows self-organization near the critical transition. Here, the authors propose a biologically inspired model in which the liquid self-organizes near the critical point. The model is composed of a reservoir of integrate-and-fire neurons. The synaptic efficacies within have unsupervised spike-time dependent plasticity (STDP). The authors propose an additional dynamical variable akin to the activity of astrocytes in the circuit, which modifies the STDP learning rates. The model self-organizes close to the critical point. The authors train several networks on the MNIST and N-MNIST dataset (a spiking version of MNIST, made for neuromorphic computing). They show that the SLM model with astrocyte-dependent STDP plasticity outperforms other LSM models.