Bio-inspired sequence learning mechanisms and theirimplementation in a memristive neuromorphic hardware
Younes Bouhadjar1,2,3,4 , Sebastian Siegel3,4 , Markus Diesmann1,5 , Rainer Waser3,6 , Emre
Neftci2,4 , Dirk J. Wouters6∗ , Tom Tetzlaff1∗
1. Institute of Neuroscience and Medicine (INM-6), & Institute for Advanced Simulation (IAS-6), & JARA BRAIN
Institute Structure-Function Relationships (INM-10), Jülich Research Centre, Jülich, Germany
2. Peter Grünberg Institute (PGI-15), Jülich Research Centre and JARA, Jülich, Germany
3. Peter Grünberg Institute (PGI-7,10), Jülich Research Centre and JARA, Jülich, Germany
4. RWTH Aachen University, Aachen, Germany
5. Department of Physics, Faculty 1, & Department of Psychiatry, Psychotherapy, and Psychosomatics, Medical School,RWTH Aachen University, Aachen, Germany
6. Institute of Electronic Materials (IWE 2) & JARA-FIT, RWTH Aachen University, Aachen, Germany*. Same orderSummary. We present a sequence learning model that explains how biological networks learn topredict upcoming elements, signal non-anticipated events, and recall sequences in response to a cue
signal. The model accounts for anatomical and electrophysiological properties of cortical neuronal
circuits and learns complex sequences in an unsupervised manner using known biological plasticity
and homeostatic control mechanisms. We further investigate the feasibility of implementing thesequence learning model on dedicated hardware mimicking brain properties, specifically focusing on
memristive crossbar arrays. Finally, we apply the model to sequence classification and anomaly
detection in streams of real-world data, and discuss the role of dendritic branches for the sequencelearning capacity.

Learning in an unsupervised manner using scarce data is a remarkable ability demonstrated by humans
and animals. This is especially evident in infants, who are able to extract meaningful patterns fromthe limited information available in their environment. The neocortex and other brain areas receiveand process this data in a sequential manner. This holds not only for sensory processing but also forhigh-level cognitive processes such as planning or reasoning. This data needs to be stored to form
predictions of upcoming events or actions in a context specific manner, and to detect non-anticipated
events. Understanding the neural mechanisms of these computations allows for the development ofenergy-efficient artificial intelligence systems with online learning capabilities.
Building on the ideas of Hawkins and Ahmad [2], we developed a biologically inspired sequence learningand prediction model (Figure 1). It learns to predict complex sequences in an unsupervised, continuous
manner using biological, local learning rules. The model can also perform probabilistic sequential memory
recall in response to ambiguous cues [3]. It strengthens the sequence learning mechanisms introducedin [2] and suggests new ones. In the following, we give a summary of these mechanisms:• Learning and storage of sequences: sequences are represented by specific subnetworks embedded
into the recurrent network. During the learning process, these subnetworks are carved out in an
unsupervised manner by a form of structural Hebbian plasticity.
• Context specificity: learning of high-order sequences is enabled by a sparse, random potential
connectivity, and by a homeostatic regulation of synaptic growth.
• Generation of predictions: neurons are equipped with a predictive state, implemented by a nonlinear
synaptic integration mimicking the generation of dendritic action potentials (dAPs).
• Mismatch detection: only a few neurons become active if a prediction matches the stimulus. In
our model, this sparsity is realized by winner-take-all (WTA) dynamics implemented in the formof inhibitory feedback. In the case of non-anticipated stimuli, the WTA dynamics cannot step in,thereby leading to a non-sparse activation of larger neuron populations.
• Sequence replay: an autonomous replay of learned sequences in response to a cue signal is enabled
by increasing neuronal excitability.
In subsequent studies, we investigate whether it is possible to implement the sequence learning algorithm on neuromorphic hardware centered around memristive devices. Our investigations suggest thatthe model is robust toward the intrinsic potentiation and depression characteristics of memristive devices
such as variability, limited precision, and synaptic failure, and shows that the memristive device can beoperated either in a binary or gradual mode without a loss in performance [4]. We further devised anelectronic circuit design of the hardware implementing the sequence learning model [5].

Task

Network structure

Network activity

Figure 1: Task: The study describes a neuronal network model that is capable of learning and processing
sequences of ordered discrete elements (such as characters, musical notes, numbers, or images). The
model can predict subsequent elements in response to the presentation of other elements after repeatedpresentation of high-order sequences with overlapping characters, and it can detect unanticipated elements
by generating a mismatch signal if the prediction is not met. Network structure: The architecture of
the model consists of a recurrent network of excitatory and inhibitory neurons. The excitatory neuron
population is divided into subpopulations with neurons that have identical stimulus preferences, and theconnections within and between these subpopulations are random and sparse. The inhibitory neurons
are mutually unconnected but each one is connected to a specific subpopulation of excitatory neurons
in a recurrent manner. During learning, sequence specific, sparsely connected subnetworks with maturesynapses are formed (light blue arrows: {‘A’, ‘D’, ‘B’, ‘E’}, dark blue arrows: {‘F’, ‘D’, ‘B’, ‘C’}).
Network activity: Snapshots of network activity upon subsequent presentation of the sequence elements‘A’ and ‘D’. When the first element ‘A’ is presented, all neurons in the corresponding subpopulations fire.
The activation of these neurons then triggers dAPs (predictions) in a subset of neurons that represent thesubsequent element ‘D’. When the next element ‘D’ is presented, only the neurons that made predictions
become active, which in turn leads to predictions in the subpopulation that represents the subsequent
subpopulation (‘B’), and so on for subsequent elements. Adapted from [1].
During the NNPC conference, we will give a summary of these findings and report on our progressin applying the sequence learning model to solve real-world tasks such as classification and detecting
detection in spiking data and the role of dendritic branches in increasing the sequence learning capacity.

References[1] Bouhadjar, Y., D. J. Wouters, M. Diesmann, and T. Tetzlaff (2022b). Sequence learning, prediction,and replay in networks of spiking neurons. PLOS Computational Biology 18 (6), e1010233.
[2] Hawkins, J. and S. Ahmad (2016). Why neurons have thousands of synapses, a theory of sequence
memory in neocortex. Frontiers in Neural Circuits 10, 23.
[3] Bouhadjar, Y., D. J. Wouters, M. Diesmann, and T. Tetzlaff (2022a). Coherent noise enables
probabilistic sequence replay in spiking neuronal networks. ArXiv , 2206.10538.
[4] Bouhadjar, Y., S. Siegel, T. Tetzlaff, M. Diesmann, R. Waser, and D. J. Wouters (2022c). Sequencelearning in a spiking neuronal network with memristive synapses. ArXiv , 2211.16592.
[5] Siegel, S., Y. Bouhadjar, T. Tetzlaff, R. Waser, R. Dittmann, and D. J. Wouters (2023). Systemmodel of neuromorphic sequence learning on a memristive crossbar array. Under review .