The brain performs many nonlinear computations through intricate spiking neural networks (SNNs). How neural network dynamics relate to arbitrary computations under these constraints is still an open question. As a strong constraint, these networks are hypothesized to be robust to perturbations and use minimal energy. The theory of Spike Coding Networks (SCNs)derives the required connectivity and dynamics for both information representation and linear dynamical systems from first principles, and achieves robustness and efficiency.Nonlinear dynamical systems have thus far only been implemented in SCNs by filtering neural inputs through sets of nonlinear dendritic basis functions.While this approach works well, it relies on providing a rich enough basis set as well as supervised trainingof the connectivity weights. Another way to implement nonlinear computations is through multiplicatively interacting synapses. However, there is currently no principled way to implement such synapses in SCNs. Here ,  we extend the core SCN derivations to implement polynomial dynamical systems, from which also the need for such multiplicatively interacting synapses arises . We demonstrate our approach with a highly accurate Lorenz attractor implementation, as well as a second-order approximation of a double pendulum. We additionally demonstrate how to implement higher-order polynomials using sequential networks with only pair-wise synapses. Finally, we derive upper bounds and expected numbers of connections based on the sparsity of the underlying representation . Overall, our work provides an alternative way to directly implement nonlinear computations in spike coding networks, and expands our understanding about the potential functions of multiplicative synapses. Furthermore, due to the high accuracy and low energy usage of our approach , this work may be of interest for neuromorphic computing.
The brain efficiently performs nonlinear computations through its intricate networks of spiking neurons, but how this is done remains elusive. While recurrent spiking networks implementing linear computations can be directly derived and easily understood (e.g., in the spike coding network framework), the connectivity required for nonlinear computations can be harder to interpret, as they require additional non-linearities (e.g. , dendritic or synaptic) weighted through supervised training. Here we extend the spike coding framework to implement any polynomial dynamical system. This results in networks requiring multiplicative synapses, which we term the multiplicative spike coding network (mSCN) . We demonstrate how the required connectivity for several nonlinear dynamical systems can be directly implemented in mSCNs, without training. We also show how to carry out higher-order polynomials with coupled networks that use only pair-wise multiplicative synapses, and provide expected numbers of connections for each synapse type . Overall, our work provides an alternative method for implementing nonlinear computations in spiking neural networks, while keeping all the attractive features of standard SCNs (such as robustness, irregular and sparse firing, and interpretable connectivity). Finally, we discuss the biological plausibility of our approach, and how the high accuracy and robustness of the approach may be of interest for neuromorphic computing.
R
performs many
efficiently performs
R
intricate spiking neural networks (SNNs). How neural network dynamics relate to arbitrary computations under these constraints is still an open question. As a strong constraint, these networks are hypothesized to be robust to perturbations and use minimal energy. The theory of Spike Coding Networks (SCNs)derives the required connectivity and dynamics for both information representation and linear dynamical systems from first principles, and achieves robustness and efficiency.Nonlinear dynamical systems have thus far only been implemented in SCNs by filtering neural inputs through sets of nonlinear dendritic basis functions.While this approach works well, it relies on providing a rich enough basis set as well as supervised trainingof the connectivity weights. Another way to implement nonlinear computations is through multiplicatively interacting synapses. However, there is currently no principled way to implement such synapses in SCNs. Here
its intricate networks of spiking neurons, but how this is done remains elusive. While recurrent spiking networks implementing linear computations can be directly derived and easily understood (e.g., in the spike coding network framework), the connectivity required for nonlinear computations can be harder to interpret, as they require additional non-linearities (e.g.
A
dendritic or synaptic) weighted through supervised training. Here
R
core SCN derivations to implement polynomial dynamical systems, from which also the need for such multiplicatively interacting synapses arises
spike coding framework to implement any polynomial dynamical system. This results in networks requiring multiplicative synapses, which we term the multiplicative spike coding network (mSCN)
R
our approach with a highly accurate Lorenz attractor implementation, as well as a second-order approximation of a double pendulum. We additionally demonstrate how to implement
how the required connectivity for several nonlinear dynamical systems can be directly implemented in mSCNs, without training. We also show how to carry out
R
using sequential networks with
with coupled networks that use
R
synapses. Finally, we derive upper bounds and
multiplicative synapses, and provide
R
based on the sparsity of the underlying representation
for each synapse type
R
way to directly implement
method for implementing
R
spike coding networks, and expands our understanding about the potential functions of multiplicative synapses. Furthermore, due to the
spiking neural networks, while keeping all the attractive features of standard SCNs (such as robustness, irregular and sparse firing, and interpretable connectivity). Finally, we discuss the biological plausibility of our approach, and how the
R
low energy usage of our approach , this work
robustness of the approach