Cortical neurons communicate with spikes, which are discrete events in time . Even if the timings of the individual events are strongly chaotic (microscopic chaos), the rate of events might still be non-chaotic or at the edge of what is known as rate chaos. Such edge-of-chaos dynamics are beneficial to the computational power of neuronal networks. We analyze both types of chaotic dynamics in densely connected networks of asynchronous binary neurons, by developing and applying a model-independent field theory for neuronal networks . We find a strongly size-dependent transition to microscopic chaos. We then expose the conceptual difficulty at the heart of the definition of rate chaos, identify two reasonable definitions, and show that for neither of them the binary network dynamics crosses a transition to rate chaos. The analysis of diverging trajectories in chaotic networks also allows us to study classification of linearly non-separable classes of stimuli in a reservoir computing approach. We show that microscopic chaos rapidly expands the dimensionality of the representation while, crucially, the number of dimensions corrupted by noise lags behind. This translates to a transient peak in the networks' classification performance even deeply in the chaotic regime, challenging the view that computational performance is always optimal near the edge of chaos. This is a general effect in high dimensional chaotic systems, and not specific to binary networks : We also demonstrate it in a continuous 'rate' network, a spiking LIF network, and an LSTM network. For binary and LIF networks , classification performance peaks rapidly within one activation per participating neuron, demonstrating fast event-based computation that may be exploited by biological neural systems, for which we propose testable predictions .
Neurons in the brain communicate with spikes, which are discrete events in time and value. Functional network models often employ rate units that are continuously coupled by analog signals. Is there a qualitative difference implied by these two forms of signaling? We develop a unified mean-field theory for large random networks to show that first- and second-order statistics in rate and binary networks are in fact identical if rate neurons receive the right amount of noise. Their response to presented stimuli, however, can be radically different. We quantify these differences by studying how nearby state trajectories evolve over time, asking to what extent the dynamics is chaotic. Chaos in the two models is found to be qualitatively different. In binary networks we find a network-size-dependent transition to chaos and a chaotic submanifold whose dimensionality expands stereotypically with time, while rate networks with matched statistics are nonchaotic. Dimensionality expansion in chaotic binary networks aids classification in reservoir computing and optimal performance is reached within about a single activation per neuron; a fast mechanism for computation that we demonstrate also in spiking networks. A generalization of this mechanism extends to rate networks in their respective chaotic regimes .
R
Cortical neurons
Neurons in the brain
R
. Even if the timings of the individual events are strongly chaotic (microscopic chaos), the rate of events might still be non-chaotic or at the edge of what is known as rate chaos. Such edge-of-chaos dynamics are beneficial to the computational power of neuronal networks. We analyze both types of chaotic dynamics in densely connected networks of asynchronous binary neurons, by developing and applying a model-independent field theory for neuronal networks . We find a strongly size-dependent transition to microscopic chaos. We then expose the conceptual difficulty at the heart of the definition of rate chaos, identify two reasonable definitions, and show that for neither of them the binary network dynamics crosses a transition to rate chaos. The analysis of diverging trajectories in chaotic networks also allows us to study classification of linearly non-separable classes of stimuli in a reservoir computing approach. We show that microscopic chaos rapidly expands the dimensionality of the representation while, crucially, the number of dimensions corrupted by noise lags behind. This translates to a transient peak in the networks' classification performance even deeply in the chaotic regime, challenging the view that computational performance is always optimal near the edge of chaos. This is a general effect in high dimensional chaotic systems, and not specific to binary
and value. Functional network models often employ rate units that are continuously coupled by analog signals. Is there a qualitative difference implied by these two forms of signaling? We develop a unified mean-field theory for large random networks to show that first- and second-order statistics in rate and binary networks are in fact identical if rate neurons receive the right amount of noise. Their response to presented stimuli, however, can be radically different. We quantify these differences by studying how nearby state trajectories evolve over time, asking to what extent the dynamics is chaotic. Chaos in the two models is found to be qualitatively different. In binary networks we find a network-size-dependent transition to chaos and a chaotic submanifold whose dimensionality expands stereotypically with time, while rate networks with matched statistics are nonchaotic. Dimensionality expansion in chaotic binary
R
: We also demonstrate it in a continuous 'rate' network, a spiking LIF network, and an LSTM network. For binary and LIF networks , classification performance peaks rapidly within one activation per participating neuron, demonstrating fast event-based computation that may be exploited by biological neural systems, for which we propose testable predictions
aids classification in reservoir computing and optimal performance is reached within about a single activation per neuron; a fast mechanism for computation that we demonstrate also in spiking networks. A generalization of this mechanism extends to rate networks in their respective chaotic regimes