The brain carries out demanding computations and learning processes with recurrent networks of spiking neurons (RSNNs)  . But computing and learning capabilities of currently available RSNN models have remained poor, especially in comparison with the performance of recurrent networks of artificial neurons, such as Long Short-Term Memory (LSTM) networks. In this article, we investigate whether deep learning can improve RSNN performance. We applied backpropagation through time (BPTT) , augmented by biologically inspired heuristics for synaptic rewiring, to RSNNs whose inherent time constants were enriched through simple models for adapting spiking neurons . We found that the resulting RSNNs approximate, for the first time , the computational power of LSTM networks on two common benchmark tasks. Furthermore, our results show that recent successes with applications of Learning-to-Learn (L2L)to LSTM networks can be ported to RSNNs. This opens the door to the investigation of L2L in data-based models for neural networks of the brain, whose activity can -- unlike that of LSTM networks -- be compared directly with recordings from neurons in the brain. In particular, L2L shows that RSNNs can learn large families of non-linear transformations from very few examples , using previously unknown network learning mechanisms. Furthermore, meta-reinforcement learning (meta-RL) shows that LSNNs can learn and execute complex exploration and exploitation strategies .
Recurrent networks of spiking neurons (RSNNs) underlie the astounding computing and learning capabilities of the brain . But computing and learning capabilities of RSNN models have remained poor, at least in comparison with ANNs. We address two possible reasons for that. One is that RSNNs in the brain are not randomly connected or designed according to simple rules, and they do not start learning as a tabula rasa network. Rather, RSNNs in the brain were optimized for their tasks through evolution, development, and prior experience. Details of these optimization processes are largely unknown. But their functional contribution can be approximated through powerful optimization methods, such as backpropagation through time (BPTT) . A second major mismatch between RSNNs in the brain and models is that the latter only show a small fraction of the dynamics of neurons and synapses in the brain. We include neurons in our RSNN model that reproduce one prominent dynamical process of biological neurons that takes place at the behaviourally relevant time scale of seconds: neuronal adaptation. We denote these networks as LSNNs because of their Long short-term memory. The inclusion of adapting neurons drastically increases the computing and learning capability of RSNNs if they are trained and configured by deep learning (BPTT combined with a rewiring algorithm that optimizes the network architecture) . In fact, the computational performance of these RSNNs approaches for the first time that of LSTM networks . In addition RSNNs with adapting neurons can acquire abstract knowledge from prior learning in a Learning-to-Learn ( L2L ) scheme, and transfer that knowledge in order to learn new but related tasks from very few examples . We demonstrate this for supervised learning and reinforcement learning .
R
The brain carries out demanding computations and learning processes with recurrent
Recurrent
A
underlie the astounding computing and learning capabilities of the brain
D
currently available
R
especially
at least
R
the performance of recurrent networks of artificial neurons, such as Long Short-Term Memory (LSTM) networks. In this article, we investigate whether deep learning can improve RSNN performance. We applied
ANNs. We address two possible reasons for that. One is that RSNNs in the brain are not randomly connected or designed according to simple rules, and they do not start learning as a tabula rasa network. Rather, RSNNs in the brain were optimized for their tasks through evolution, development, and prior experience. Details of these optimization processes are largely unknown. But their functional contribution can be approximated through powerful optimization methods, such as
R
, augmented by biologically inspired heuristics for synaptic rewiring, to RSNNs whose inherent time constants were enriched through simple models for adapting spiking neurons
. A second major mismatch between RSNNs in the brain and models is that the latter only show a small fraction of the dynamics of neurons and synapses in the brain. We include neurons in our RSNN model that reproduce one prominent dynamical process of biological neurons that takes place at the behaviourally relevant time scale of seconds: neuronal adaptation. We denote these networks as LSNNs because of their Long short-term memory. The inclusion of adapting neurons drastically increases the computing and learning capability of RSNNs if they are trained and configured by deep learning (BPTT combined with a rewiring algorithm that optimizes the network architecture)
R
We found that
In fact, the computational performance of these RSNNs approaches for
R
resulting RSNNs approximate, for the first time , the computational power of LSTM networks on two common benchmark tasks. Furthermore, our results show that recent successes with applications of Learning-to-Learn (L2L)to LSTM networks can be ported to RSNNs. This opens the door to the investigation of L2L in data-based models for neural networks of the brain, whose activity can -- unlike
first time
R
-- be compared directly with recordings from neurons in the brain. In particular,
. In addition RSNNs with adapting neurons can acquire abstract knowledge from prior learning in a Learning-to-Learn (
R
shows that RSNNs can learn large families of non-linear transformations
) scheme, and transfer that knowledge in order to learn new but related tasks
R
, using previously unknown network learning mechanisms. Furthermore, meta-reinforcement learning (meta-RL) shows that LSNNs can learn and execute complex exploration and exploitation strategies
. We demonstrate this for supervised learning and reinforcement learning