Neurons communicate with spikes, which are discrete events in time. Functional network models often employ rate units that are continuously coupled by analog signals. Is there a benefit of discrete signaling? By a unified mean-field theory we show that large random networks of rate and binary units have identical second order statistics. Yet their stimulus processing properties are radically different: We discover a chaotic sub-manifold in binary networks that does not exist in rate models. Its dimensionality increases with time after stimulus onset and reaches a fixed point depending on the synaptic coupling strength. Low dimensional stimuli are transiently expanded into higher-dimensional representations within this manifold. High noise resilience persists not only near the edge of chaos, but throughout the chaotic regime. In rate models of spiking activity, the effective spiking noise suppresses chaos, severely impairing classification performance. Chaotic rate networks without effective spiking noise also show the transient performance boost. The transitions to chaos in the two models do not coincide and have qualitatively different causes. Our theory mechanistically explains these observations. These findings have several implications. 1) Discrete state networksreach optimal performance with weaker synapses; implying lower energetic costs for synaptic transmission. 2) The classification mechanism is robust to noise, compatible with fluctuations in biophysical systems . 3) Optimal performance is reached after only a single activation per participating neuron ; demonstrating event-based computation with short latencies. 4) The chaotic sub-manifold predicts a transient increase of variability after stimulus onset. Our results thus provide a hitherto unknown link between recurrent and chaotic dynamics of functional networks, neuronal variability, and dimensionality of neuronal responses .
Cortical neurons communicate with spikes, which are discrete events in time. Even if the timings of the individual events are strongly chaotic (microscopic chaos), the rate of events might still be non-chaotic or at the edge of what is known as rate chaos. Such edge-of-chaos dynamics are beneficial to the computational power of neuronal networks. We analyze both types of chaotic dynamics in densely connected networks of asynchronous binary neurons, by developing and applying a model-independent field theory for neuronal networks. We find a strongly size-dependent transition to microscopic chaos. We then expose the conceptual difficulty at the heart of the definition of rate chaos, identify two reasonable definitions, and show that for neither of them the binary network dynamics crosses a transition to rate chaos . The analysis of diverging trajectories in chaotic networks also allows us to study classification of linearly non-separable classes of stimuli in a reservoir computing approach. We show that microscopic chaos rapidly expands the dimensionality of the representation while, crucially, the number of dimensions corrupted by noise lags behind. This translates to a transient peak in the networks' classification performance even deeply in the chaotic regime, challenging the view that computational performance is always optimal near the edge of chaos. This is a general effect in high dimensional chaotic systems, and not specific to binary networks: We also demonstrate it in a continuous 'rate' network, a spiking LIF network, and an LSTM network. For binary and LIF networks, classification performance peaks rapidly within one activation per participating neuron , demonstrating fast event-based computation that may be exploited by biological neural systems, for which we propose testable predictions .
R
Neurons
Cortical neurons
R
Functional network models often employ rate units that are continuously coupled by analog signals. Is there a benefit of discrete signaling? By a unified mean-field theory we show that large random networks of rate and binary units have identical second order statistics. Yet their stimulus processing properties are radically different: We discover a chaotic sub-manifold in binary networks that does not exist in rate models. Its dimensionality increases with time after stimulus onset and reaches a fixed point depending on the synaptic coupling strength. Low dimensional stimuli are transiently expanded into higher-dimensional representations within this manifold. High noise resilience persists not only near the edge of chaos, but throughout the chaotic regime. In rate models of spiking activity,
Even if the timings of the individual events are strongly chaotic (microscopic chaos), the rate of events might still be non-chaotic or at the edge of what is known as rate chaos. Such edge-of-chaos dynamics are beneficial to the computational power of neuronal networks. We analyze both types of chaotic dynamics in densely connected networks of asynchronous binary neurons, by developing and applying a model-independent field theory for neuronal networks. We find a strongly size-dependent transition to microscopic chaos. We then expose the conceptual difficulty at the heart of
R
effective spiking noise suppresses chaos, severely impairing classification performance. Chaotic rate networks without effective spiking noise also show the transient performance boost. The transitions to chaos in the two models do not coincide and have qualitatively different causes. Our theory mechanistically explains these observations. These findings have several implications. 1) Discrete state networksreach optimal performance with weaker synapses; implying lower energetic costs for synaptic transmission. 2) The classification mechanism is robust to noise, compatible with fluctuations in biophysical systems
definition of rate chaos, identify two reasonable definitions, and show that for neither of them the binary network dynamics crosses a transition to rate chaos
R
3) Optimal performance is reached after only a single
The analysis of diverging trajectories in chaotic networks also allows us to study classification of linearly non-separable classes of stimuli in a reservoir computing approach. We show that microscopic chaos rapidly expands the dimensionality of the representation while, crucially, the number of dimensions corrupted by noise lags behind. This translates to a transient peak in the networks' classification performance even deeply in the chaotic regime, challenging the view that computational performance is always optimal near the edge of chaos. This is a general effect in high dimensional chaotic systems, and not specific to binary networks: We also demonstrate it in a continuous 'rate' network, a spiking LIF network, and an LSTM network. For binary and LIF networks, classification performance peaks rapidly within one
R
; demonstrating
, demonstrating fast
R
with short latencies. 4) The chaotic sub-manifold predicts a transient increase of variability after stimulus onset. Our results thus provide a hitherto unknown link between recurrent and chaotic dynamics of functional networks, neuronal variability, and dimensionality of neuronal responses
that may be exploited by biological neural systems, for which we propose testable predictions