Artificial Neural Microcircuits as Building Blocks for
Neuromorphic Systems
Andrew Walter1 , Shimeng Wu1 , Andy M. Tyrrell1 , Liam McDaid2 , Malachy McElholm2 ,Nidhin Thandassery Sumithran2 , Jim Harkin2 , Martin A. Trefzer1
1. University of York, School of Physics, Engineering and Technology, UK
2. Ulster University, School of Computing, Engineering and Intelligent Systems, UK

Summary. Artificial Neural Networks (ANNs) are the most widely employed forms of neuromorphiccomputation. However, unlike the biological systems they draw inspiration from, the current standard
is for ANNs to be structurally homogeneous. This is at odds with research evidencing that structure
plays a significant role in supporting learning capability and function of the brain. In this paper,an alternative approach inspired by biological Neural Microcircuits is suggested. The conceit ofassembling ANNs using a catalogue of Artificial Neural Microcircuits, and a methodology to assemble
said catalogue, is articulated; before showing the results of initial work to produce such a catalogue.

Introduction
The current state-of-the-art for ANNs is focused on large, topologically homogeneous feed forward orrecurrent networks which are tailored for specific applications either via machine learning or evolutionary
algorithms [7] [2]. Some less widely used alternatives do exist, generally focused on the use of evolutionary
approaches to produce complex topologies from scratch [8].
Advances in neuroscience paint a picture of biological nervous systems that possess a much morecompartmentalised architecture; built from various “fundamental processing units” called Neural Microcircuits, which work together both hierarchically and in parallel [3, 6].
This work presents the first steps towards drawing more inspiration from the latter kind of architecture:articulating a means of employing novelty search to produce a catalogue of Artificial Neural Microcircuits,
which can then be used as “off-the-shelf” components to fashion larger, application specific Spiking NeuralNetworks.

Generating Artificial Neural Microcircuits
To assemble a catalogue of Microcircuits, it is necessary to: (i) generate a selection of candidate Microcircuits; (ii) assess them based on some criteria; and (iii) use the assessment to select which Microcircuits
would be useful components. An overview of this flow is shown in figure 1, and is elaborated upon below.
In order to allow the Microcircuits in the catalogue to be somewhat substrate independent, rather
than assembling possible Microcircuits out of discreet neurons, Neural Circuit Motifs are used instead;allowing for the injection of some additional biological domain knowledge [1, 6]. Several of these motifs
are then combined to form larger Microcircuits, which are represented using a trio of connection matrices:(i) one for external inputs; (ii) one for internal connections; and (iii) a final array for outputs.
Microcircuits are assessed by feeding them a stimulus pattern, or a set of stimulus patterns, producing
outputs in the form of spike trains. By comparing these spike trains, though the use of the Bivariate

Figure 1: The methodology used to generate the catalogue of Artificial Neural Microcircuits.

Figure 2: Left: Input text encoding 466. Right: Illustration of Microcircuit 466.
SPIKE-Distance metric [4], it is possible to build up a matrix of behavioural difference measurements.
These values can then be used to calculate a novelty value for each Microcircuit, allowing for a Novelty
Search algorithm [5] to be used to select which individuals are to be added to the Microcircuit catalogue.

Initial Experiments
To test the methodology, a test run was performed, using spike trains derived from UTF-8 character
encoding of a text sample as 8-input stimulus (shown on the left of figure 2). From a total pool of 5,000
Microcircuits, the novelty search function transferred 50 to the catalogue. The behaviours of these 50
were investigated to determine what functionality they possessed, in particular whether their outputs
showed a strong correlation to a given patterns of inputs. While a variety of behaviours occurred, a clear
validation of the concept can be found in Microcircuit 466, shown on the right of figure 2.
Produced in the fifth iteration, with 9 neurons across 3 motifs, Microciruit 499 displayed a very strong
correlation with the four punctuation mark characters in the sample text used (Space, Comma, Dash andStop)—without having explicitly searched for this functionality. A Microcircuit with such a behaviourhas a clear use as a component of a larger Neuromorphic system. It is easy to see how a catalogue ofsimilar Microcircuits could be used to assemble a network able to completely parse UTF-8 text streams,
but of course it’s also possible to go further.
The exact same Microcircuits could be used as part of a network that takes in handwritten characters,with other kinds of Microcircuits forming the interface between the character images and the character
recognisers; or in a completely different application where the 8-bit patterns they recognise are someother stimuli of interest, providing a set of versatile building blocks with innate capabilities, with no needto train them, for text processing.

References[1] J. Byrne. Intro to neurons and neuronal networks, 1999. https://nba.uth.tmc.edu/neuroscience.
[2] S. Ding, H. Li, C. Su, J. Yu, and F. Jin. Evolutionary artificial neural networks: a review. Artificial
Intelligence Review, 39:251–260, 2013.
[3] S. Grillner and A. Graybiel. Microcircuits: the Interface Between Neurons and Global Brain Function.
MIT Press, 2006.
[4] T. Kreuz, D. Chicharro, M. Greschner, and R. Andrzejak. Time-resolved and time-scale adaptive
measures of spike train synchrony. Journal of Neuroscience Methods, 195:92–106, 2010.
[5] J. Lehman and K. Stanley. Tabandoning objectives: Evolution through the search for novelty alone.
Evolutionary Computation, 19:189–223, 2011.
[6] L. Luo. Principles of Neurobiology. Garland Science, 2016.
[7] J. Schmidhuber. Deep learning in neural networks: An overview. Neural Networks, 61:85–117, 2015.
[8] K. Stanley and R. Miikkulainen. Evolving neural networks through augmenting topologies. Evolutionary Computation, 10:99–127, 2002.