Increasing Liquid State Machine Performance with Edge-of-Chaos Dynamics Organized by Astrocyte-modulated Plasticity

1 Introduction . With the recent rise of neuromorphic [ 1–4 ] and edge computing [ 5 , 6 ] , the liquid state machine ( LSM ) learning framework [ 7 ] has become an attractive alternative [ 8–11 ] to deep neural networks owing to its compatibility with energy-efficient neuromorphic hardware [ 12–14 ] and inherently low training complexity . Originally proposed as a biologically plausible model of learning , LSMs avoid training via backpropagation by using a sparse , recurrent , spiking neural network ( liquid ) with fixed synaptic connection weights to project inputs into a high dimensional space from which a single neural layer can learn the correct outputs . Yet , these advantages over deep networks come at the expense of 1 ) sub-par accuracy and 2 ) extensive data-specific hand-tuning of liquid weights . Interestingly , these 1Code and data available at https : //github.com/combra-lab/NALSM 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . two limitations have been targeted by several studies that tackle one [ 15 , 16 ] or the other [ 17 , 18 ] , but not both . This has limited the widespread use of LSMs in real-world applications [ 8 ] . In that sense , there is an unmet need for a unified , brain-inspired approach that is directly applicable to the emerging neuromorphic and edge computing technologies , facilitating them to go mainstream . As a general heuristic , LSM accuracy is maximized when LSM dynamics are positioned at the edgeof-chaos [ 19–21 ] and specifically in the vicinity of a critical phase transition [ 22–25 ] that separates : 1 ) the sub-critical phase , where network activity decays , and 2 ) the super-critical ( chaotic ) phase , where network activity gets exponentially amplified . Strikingly , brain networks have also been found to operate near a critical phase transition [ 26–28 ] that is modeled as a branching process [ 25 , 26 ] . Current LSM tuning methods organize network dynamics at the critical branching factor by adding forward and backward communication channels on top of the liquid [ 15 , 16 ] . This , however , results in significant increases in training complexity and violates the LSM ’ s brain-inspired self-organization principles . For example , these methods lack local plasticity rules that are widely observed in the brain and considered a key component for both biological [ 29 ] and neuromorphic learning [ 3 , 2 , 4 ] . A particular local learning rule , spike-timing-dependent plasticity ( STDP ) , is known to improve LSM accuracy [ 17 , 18 ] . Yet , current methods of incorporating STDP into LSMs further exacerbate the limitations of data-specific hand-tuning as they require additional mechanisms to compensate for the STDP-imposed saturation of synaptic weights [ 17 , 30–33 ] . This signifies the scarcity of LSM tuning methods that are both computationally efficient and data-independent . A long-neglected non-neuronal cell in the brain , astrocytes , is now known to play key roles in modulating brain networks [ 34–39 ] , from modifying synaptic plasticity [ 40–42 ] to facilitating switching between cognitive states [ 43–46 ] that have been linked to a narrow spectrum of dynamics around the critical phase transition [ 47–51 ] . The mechanisms that astrocytes use to modulate neurons include the integration of the activity of thousands of synapses into a slow intracellular continuous signal that feeds back to neurons by affecting their synaptic plasticity [ 52–55 , 42 ] . The unique spatiotemporal attributes [ 56 , 57 ] identified in astrocytes align well with the brain ’ s remarkable ability to self-organize its massive and highly recursive networks near criticality . That is why astrocytes present a fascinating possibility of forming a unified feedback modulation mechanism required to improve baseline LSM accuracy while eliminating data-specific hand-tuning . Here , we propose the neuron-astrocyte liquid state machine ( NALSM ) , where a biologically inspired astrocyte model organized liquid dynamics near a critical phase transition , by modulating STDP . We show that NALSM combined the computational benefits of both STDP and critical branching dynamics by demonstrating its accuracy advantage compared to other LSM methods on two datasets : 1 ) MNIST [ 58 ] , and 2 ) N-MNIST [ 59 ] . We demonstrate that , similar to its biological counterpart that handles new and unstructured information with robustness and versatility , NALSM maintains the state-of-the-art LSM performance without re-tuning training parameters for each tested dataset . We also show that a NALSM with a large enough liquid can attain comparable accuracy to fully-connected multi-layer spiking neural networks trained via backpropagation on 1 ) MNIST [ 58 ] , 2 ) N-MNIST [ 59 ] , as well as 3 ) Fashion-MNIST [ 60 ] . Our results suggest that the under-performance and high training difficulty of current neuromorphic methods can be addressed by harvesting neuroscience knowledge and further translating biological principles to computational mechanisms . 2 Methods . 2.1 The neuron-astrocyte liquid state machine . To construct the NALSM , we started with a baseline LSM model consisting of 2 layers : 1 ) a spiking liquid , and 2 ) a linear output layer . Next , we added STDP to the LSM liquid , forming the LSM+STDP model . We developed a biologically faithful leaky-integrate-and-modulate ( LIM ) astrocyte model , which we embedded in the LSM+STDP liquid , to form the NALSM . The process is formalized below . LSM Model We implemented the baseline LSM as a 3-dimensional neural network ( liquid ) consisting of 1 , 000 neurons surrounded by 1-dimensional layers of input and output neurons . Number of input neurons was 784 and 2 , 312 for MNIST and N-MNIST , respectively ( See Appendix A.1 ) . We used the leaky-integrate-and-fire ( LIF ) model [ 3 ] for input and liquid neurons , modeled as : dvi dt = − 1 τv vi ( t ) + ui ( t ) − θiσi ( t ) ( 1 ) ui ( t ) = ∑ j 6=i wij ( αu ∗ σj ) ( t ) + bi ( 2 ) where vi is the membrane potential and ui is the synaptic response current of neuron i , θi is the membrane potential threshold , σi ( t ) = ∑ k δ ( t − tki ) is the spike train of neuron i with tki being the time of the k-th spike , wij is the weight connecting neuron j to i , bi is the bias of neuron i , and αu ( t ) = τ−1u exp ( −t/τu ) H ( t ) is the synaptic filter with H ( t ) being the unit step function ( See Appendix A.2 ) . All LIF neurons had a 2 ms absolute refractory period . Liquid neurons were excitatory and inhibitory with 80 % /20 % ratio . Input neurons did not have an excitatory/inhibitory distinction and had random excitatory and inhibitory connections to liquid neurons . From here on , we will refer to connections between input neurons and liquid neurons as IL connections , inter-liquid connections as LL , and liquid to output connections as LO . In line with [ 7 ] , we created LL connections using probabilities based on Euclidean distance , D ( i , j ) , between any two neurons i , j : P ( i , j ) = C · exp ( − ( D ( i , j ) λ ) 2 ) ( 3 ) with closer neurons having higher connection probability . Parameters C and λ set the amplitude and horizontal shift , respectively , of the probability distribution ( See Appendix A.3 ) . Density of IL connections was 15 % . The output layer was a dense layer consisting of 10 linear neurons . LSM+STDP Model We added unsupervised , local learning to the LSM model by letting STDP change each LL and IL connection [ 61 ] , modeled as : dw dt = A+Tpre ∑ o δ ( t− topost ) −A−Tpost ∑ i δ ( t− tipre ) ( 4 ) where A+ = A− = 0.15 are the potentiation/depression learning rates and Tpre/Tpost are the pre/post-synaptic trace variables , modeled as , τ∗+ dTpre dt = −Tpre + a+ ∑ i δ ( t− tipre ) ( 5 ) τ∗− dTpost dt = −Tpost + a− ∑ o δ ( t− topost ) ( 6 ) where a+ = a− = 0.1 are the discrete contributions of each spike to the trace variable , τ∗+ = τ ∗ − = 10 ms are the decay time constants , tipre and t o post are the times of the pre-synaptic and post-synaptic spikes , respectively . We constrained connection weights to : 1 ) IL : [ −3 , 3 ] , 2 ) excitatory LL : [ 0 , 3 ] , and 3 ) inhibitory LL : [ −3 , 0 ] . We used the same STDP parameters for all models and experiments . LIM Astrocyte Model We developed the astrocyte model as a leaky integrator with a continuous output value Aastro− , expressed as : τasto dAastro− dt = −Aastro− + wastro ∑ i∈Nliq δ ( t− ti ) − wastro ∑ j∈Ninp δ ( t− tj ) + bastro ( 7 ) where Aastro− directly mapped to A− in equation ( 4 ) , bastro = A+ adjusted the astrocyte output to the fixed STDP potentiation learning rate , Nliq and Ninp are the sets of liquid and input neurons , respectively , andwastro set astrocyte responsiveness to network activity ( See Appendix A.4 ) . Ignoring the decay and bias terms , the astrocyte model computed the difference in the number of spikes produced by liquid neurons and input neurons . Functionally , this is equivalent to computing the ratio of spikes emitted by the liquid over the input neurons : BFproxy ( t ) = ∑ i∈Nliq δ ( t− ti ) ∑ j∈Ninp δ ( t− tj ) ( 8 ) Specifically , when the liquid produced more spikes than the input neurons , their difference was positive which translated to BFproxy > 1.0 , and vice versa . This approach to measure liquid dynamics acted as a network level approximation of the branching factor , σBF , which is normally evaluated for each neuron ( See 2.3.1 ) . We empirically confirmed that BFproxy = 1.0 aligned with the critical branching factor , σBF = 1.0 ( Fig . 1 B ) . Hence , as dynamics became progressively supercritical ( σBF > 1.0 ) , BFproxy became greater than 1 , which caused the LIM astrocyte to increase STDP depression learning rate above the fixed STDP potentiation learning rate ( Aastro− → A− > A+ ) . This caused STDP to decrease the average weight of LL and IL connections , which decreased number of spikes produced by the liquid and made dynamics less super-critical ( Fig . 1 B ) . The reverse occurred as dynamics became progressively sub-critical . As a result of astrocyte modulation , liquid dynamics oscillated between sub-critical and super-critical until eventual stabilization near the critical branching factor ( See Appendix A.4 ) . NALSM Model We completed NALSM by adding the LIM astrocyte to the LSM+STDP model ’ s liquid ( Fig . 1 A ) . As described above , the LIM astrocyte integrated activity from input and liquid neurons , and continuously controlled the STDP depression learning rate . 2.2 Training . Model training was done in 3 steps : 1 ) initialization of IL and LL liquid connections , 2 ) passing all data through the liquid resulting in liquid neuron spike counts , and 3 ) training the output layer on the spike counts . The steps are further detailed below .

Summary: This paper presents a novel type of liquid state machine - neuron-astrocyte liquid state machine (NALSM), which introduces feedback from the network activity to the synaptic plasticity. The proposed mechanism helps to position the liquid (reservoir) layer in the edge-of-chaos dynamics regime, a known key factor to achieve good computational performance. The main contribution of this paper is transforming a biological observation (the role played by astrocytes for modulating the synaptic plasticity) into a novel idea of modulating the liquid activity. It uses feedbacks to obtain edge-of-chaos property in liquid state machine (LSM), and further showing meaningful improvement over the previously proposed LSM-based methods for real-world tasks.