Journal of Robotics, Networking and Artificial Life
Vol. 7(1); June (2020), pp. 63–67
DOI: https://doi.org/10.2991/jrnal.k.200512.013; ISSN 2405-9021; eISSN 2352-6386
https://www.atlantis-press.com/journals/jrnal

Research Article

Neuromorphic Computing in Autoassociative Memory witha Regular Spiking Neuron Model
Naruaki Takano1,*, Takashi Kohno2
Graduate School of Information Science and Technology, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Hongo, Bunkyo-ku, Tokyo 113-8656, Japan
Institute of Industrial Science, The University of Tokyo, 4-6-1, Komaba, Meguro-ku, Tokyo 153-8505, Japan12ARTICLE INFOABSTRACT

Article History

Digital Spiking Silicon Neuron (DSSN) model is a qualitative neuron model specifically designed for efficient digital circuit
implementation which exhibits high biological plausibility. In this study we analyzed the behavior of an autoassociative memory
composed of 3-variable DSSN model which has a slow negative feedback variable that models the effect of slow ionic currents
responsible for Spike Frequency Adaptation (SFA). We observed the network dynamics by altering the strength of SFA which
is known to be dependent on Acetylcholine volume, together with the magnitude of neuronal interaction. By altering theseparameters, we obtained various pattern retrieval dynamics, such as chaotic transitions within stored patterns or stable and highretrieval performance. In the end, we discuss potential applications of the obtained results for neuromorphic computing.

Received 06 November 2019
Accepted 16 March 2020
Keywords
Spiking neural network
associative memory
DSSN model
spike frequency adaptation© 2020 The Authors. Published by Atlantis Press SARL.
This is an open access article distributed under the CC BY-NC 4.0 license (http://creativecommons.org/licenses/by-nc/4.0/).

1. INTRODUCTION
Neuromorphic computing is an attempt to realize a computing architecture capable of complex and robust calculations with low power
consumption, which is designed by either imitating or inspired by
the information processing mechanisms in the nervous system.
Unlike the conventional von Neumann computer architecture inwhich CPU and memory units exist separately, the neuromorphic hardware generally has highly distributed computation andmemory units. Such a massively parallelized computation whichovercomes the von Neumann bottleneck is expected to reduceoverall energy consumption.
Silicon Neuronal Networks (SiNNs) are a neuromorphic hardwarethat attempt to reproduce the electrophysiological activities in thenervous system by electronic circuits. Models of neurons, synaptic
activities and designs of underlying algorithms are crucial factors
as the components in the SiNNs. Many neuron models have been
proposed for reproducing various traits of neurons. However, theygenerally undergo the problem of trade-off between implementation cost and biological plausibility [1].
Digital Spiking Silicon Neuron (DSSN) model is a qualitative
neuron model which reproduces dynamical structure in various
neuronal firing activities and is designed to be implemented efficiently on digital circuit [2].
As a fundamental model of neuromorphic computation, an allto-all connected network composed of DSSN model has been
Corresponding author. Email: naruakitakano@sat.t.u-tokyo.ac.jp*

implemented and autoassociative task was performed [3]. The
network retrieved a stored pattern given corresponding corrupted input patterns, and its performance varied greatly oneach neuron’s parameters. It was shown that the performanceis dependent on the class of the neuron model (Class 1 and 2 in
Hodgkin’s Classification).
In our study we adopt the equivalent system, however, we incorporate Regular Spiking class for each neuron. Regular Spiking classexhibit Spike Frequency Adaptation (SFA) which is a convergenceof firing frequency from high value to a low value given a step current with sufficiently large magnitude.
Moreover, the strength of SFA is known to be reduced by cholinergic modulation [4] and it has been modeled on network ofHodgkin Huxley type neuron model [5,6]. We aim to apply andutilize the modulatory effect for neuromorphic computing. For thispurpose, we studied network dynamics by altering corresponding
parameters of the DSSN model.

2. NEURON AND SYNAPSE MODEL
Two variables in the DSSN model, membrane potential v and avariable n that represents a group of ion channels, are responsible
for generating neuronal spikes.
Spike frequency adaptation is reproduced by introducing an additional slow negative feedback variable q, which results to 3-variable
DSSN model [7]. For simplicity, we adopt the following 3-variable
DSSN model with linear q-nullcline [7] instead of the nonlinear
q-nullcline model [8].

N. Takano and T. Kohno / Journal of Robotics, Networking and Artificial Life 7(1) 63–67

64

dv f
= ( f (v ) - n - q + I 0 + I stim ) ,dt t
dn 1		 (1)= ( g (v ) - n),
dt t
dq e
= (v - v0 - a q),
dt t
where,
 a (v − b fn )2 + c fn (v < 0)		 f (v ) =  fn
(2)2a fp (v − b fp ) + c fp (v ≥ 0),			

2
 a gn (v − bgn ) + c gn (v < v g )g (v ) = 2
a gp (v − bgp ) + c gp (v ≥ v g ).

(3)

Parameters afn, bfn, cfn, afp, bfp, cfp are tuned to reproduce the v nullcline.
Similarly, parameters agn, bgn, cgn, agp, bgp, cgp determine the shape of then nullcline. Parameter I0 is a constant that model bias current and Istim
represents input stimulus. Since DSSN model is a qualitative model,
all variables are dimensionless except t which is modeled such thatits value is set approximately at a timescale of seconds. Parameters f,t and e control time constants of variables v, n and q. In particular,
e controls decay time of adaptation dynamics.
Parameter a controls strength of SFA, which is defined as the difference of Inter-spike-Interval (ISI) between the initial and theconverged state. When a is low, the effect of SFA is strong (seeFigure 1). Since SFA is known to be reduced during cholinergic
modulation, magnitude of a can be mapped to concentration ofAcetylcholine during modulation.
Synaptic current of each neuron is modeled as,dI s a syn (1 − I s )=		 dt  − b syn I s(v > 0)(v ≤ 0),(4)where c is a parameter that denotes the magnitude of neuronalinteraction and Iext is a constant bias current.
Four mutually orthogonal patterns (Figure 2a) were stored in thenetwork by configuring synaptic weight matrix W by the following
correlation rule,1 p
wu xiu x uj (i ≠ j)
Wi , j =  p ∑u =1		0(i = j) ( p = 4, 1 ≤ i, j ≤ 256).

Here, x is a vector of size 256, where pixels in a pattern (16 × 16 pixels)
are binary coded: +1 for black pixel and −1 for white pixel. wu encodes
weight bias of each pattern which are set as wu = 1.0 (u = 1, 2, 3, 4) inthe following simulations unless explicitly noted. Based on the storedpatterns, we produced sets of corrupted input patterns, where certain
percentage of randomly selected pixels are inverted. Figure 2b shows
an example of sets of input data computed based on Pattern 1.
The autoassociative task is performed as follows: First, we initialized the network state with corrupted input data by injecting stepcurrent Iext = 0.15 for 0.5 s (1334 timesteps) to the neurons that
encode black pixel. Then, we stimulated all the neurons with Iext =0.15 to evoke repetitive neuronal activity.
We evaluated the network state by introducing Phase Syn­chronization Index (PSI) which quantifies the level of synchronization, and an overlap index Mu which quantifies the similarity
between firing times of every neuron and uth stored pattern. Basedon the firing phase in the repetitive firing activity, PSI and overlap
between the network output is computed by,		 PSI(t ) =andwhere asyn = 83.3 and bsyn = 333.3.

(6)		 Mu (t ) =1 N∑ exp(ifj (t )) , 
N j =1(7)

1 N u∑x exp(ifj (t )) , N j =1 j

(8)

3. NETWORK CONFIGURATION
AND METHOD

(i: imaginary number)

Our network model is an all-to-all connected network of 256 neurons (3-variable DSSN model), and synaptic efficacy in each synapse is given by synaptic weight matrix W. External input to ithneuron is given as,		 fj (t ) = 2pk + 2p		   IistimN= c ∑W I + I , j =1ji, j siext(5)

where firing phase of neuron j is defined as,t − t kjtk +1
j− t kj.(tk:kth firing time of neuron j)
(a)

(b)

Figure 1 | Spike frequency adaptation observed in DSSN model for a =0.05 (Top) and 0.1 (Bottom).

(9)

Figure 2 | (a) Stored patterns. (b) Input data (based on Pattern 1).

N. Takano and T. Kohno / Journal of Robotics, Networking and Artificial Life 7(1) 63–67

4. SIMULATION RESULTS
4.1. Weak Neuronal Interaction
We obtained stable overlap dynamics and high retrieval performance at the weak neuronal interaction region (c = 0.005).
Figure 3 shows a successful retrieval process of Pattern 1. Figure 4
shows the retrieval performance evaluated over 100 trials for eachvalue of a. Each trial was performed until t = 10. For a = 0.05 and0.1, the successful retrieval rate keeps higher than 90% when theerror rate is equal to 30% or less, which is comparable with networkof class 2 neurons [3].
The superiority of retrieval performance can be reasoned bybiphasic (Type 2) Phase Response Curve (PRC) [9] of neurons(Figure 5). It is known that slow negative feedback current isresponsible for Type 2 PRC [5]. In addition, it is commonly
known that most class 2 neurons have Type 2 PRC. These results
support the comparable performance between network of class
2 neurons and our model.

65

Phase response curve of Regular Spiking class neuron was computedby applying a perturbation only after the adaptation has fully stabilized to determine a unique firing period. As a result, we found thatthe zero-phase-shift-crossing point is essentially altered by a whichin effect alters the overall retrieval performance. In other words, wemay be able to design optimal PRC with a for retrieval tasks.
Magnitude of phase shift (ΔF) may not be an essential factor of theperformance since increasing the magnitude of neuronal interaction did not give significant improvements (Figure 6).

4.2. Moderate Neuronal Interaction
We observed chaotic transition within stored patterns (Chaoticitineracy) when we set moderate neuronal interaction (c = 0.05).
Figure 7a shows overlap dynamics with different a. Figure 7b is thestaying period distribution without distinguishing patterns whichwere retrieved for a certain time. Maximum staying period wasextended by decreasing a.
Previous study with the discrete Hopfield Network showed that
adaptation term has a similar effect as a temperature term which isFigure 3 | Overlap dynamics in a successful retrieval process.
Figure 6 | Retrieval performance at c = 0.005 and 0.01 (a = 0.2).

(a)Figure 4 | Retrieval performance. x-axis shows error rate of the input
pattern. y-axis is the percentage of successful retrieval. Successful retrieval
was defined when PSI and overlap of corresponding input pattern
exceeded 0.9.

Figure 5 | Phase response curve of 3-variable DSSN model.

(b)

Figure 7 | (a) Overlap for c = 0.05. (Top) a = 0.1, (Bottom): a = 0.07.
(b) Duration distribution (ISI-Normalized) of the pattern retrieval. Averaged
from 10 trials of t = 1000 simulation and 40 bins. Retrieval is defined whenoverlap exceeded 0.8. Dashed blue line shows when count = 1.0.

66

N. Takano and T. Kohno / Journal of Robotics, Networking and Artificial Life 7(1) 63–67

observed at different parameter regions of c and a. By setting small
c value, the effect of neuronal interaction may be considered as perturbations applied to each neuron, therefore evoking a characteristic of PRC: Type 2 PRC caused from the slow feedback current led
to high retrieval performance. On the other hand, the synchronous
network behavior switches to chaotic itineracy when c is increased.
In this region, it can be assumed that destabilization effect caused
by SFA [6] becomes more significant.
Figure 8 | Retrieval count of strong attractor and average count of weak
attractors. Average computed from 40 trials.

Figure 9 | Stability matrix computed from gradient of average divergence(log scale) between fiducial trajectories and 10 perturbed trajectories foreach fiducial trajectory. Color bar shows gradient value.

known to destabilize attractors [6]. Similarly, in our model, variable
q and SFA strength a is assumed to be responsible for destabilization of attractors leading to chaotic itineracy.
We then introduced heterogeneity in the network by grading eachstored pattern with weight bias wu. “Strong attractors” and “weakattractors” were distinguished by the size of wu. We set w1 = 1.03
and w2,3,4 = 1.0 (Pattern 1 corresponds to the strong attractor andthe others weak). Then, we counted how often each pattern wasretrieved (when overlap for the pattern exceeded 0.8) and observed
the preference of attractor strength being modulated by a (Figure 8).
Strong preference for strong attractor is observed at a = 0.1,
whereas the preference is relaxed as a is decreased.

To model the level of Acetylcholine volume, we then investigateda-dependent effects at the above distinct regions resulting in rich
dynamics. Although some biological correspondence, especiallyon the modulation of attractor strength preference (Figure 8) isdiscussed in the prior research [6], biological plausibility of ournumerical simulation results such as duration distribution of theretrieval (Figure 7) need to be discussed in future.
From the engineering perspective, it is known that all-to-all connected networks are applicable to solving optimization problems[10]. Moreover, chaotic dynamics can be utilized to escape the local
minima and obtain the global minima [11]. Similarly, our spiking
neural network may be developed to solve optimization problems
on low power hardware in the future. All results in this work wereobtained by floating-point operations on software simulation(Euler method dt = 0.000375). One of the future works is to implement our models in fixed-point operations on FPGA devices.

APPENDIX
Table 1 | Parameter set for 3-variable DSSN model
Par.

Val.

Par.

Val.

Par.

Val.

afn
afp
agn
agp
te

8.0
−8.0
4.0
16.0
2−90.03

bfn
bfp
bgn
bgp
f
v0

−0.25
0.25
−2−4 - 2−5
2−5 - 2−2
0.625
−0.41

cfn
cfp
cgn
cgp
rg
I0

−0.5
0.5
−0.77083333−0.6875−0.26041666−0.09

Par., Parameter; Val., Value. (v0 and e take original values in this paper.)4.3. Transition of Dynamical Behavior

CONFLICTS OF INTEREST

Effects of c and a on the network’s dynamical behavior is analyzed
in the following stability matrix (Figure 9). Each value of the matrix
is computed by calculating the gradient of the log-scaled Euclidean
distance between fiducial trajectories (vector of v, n, q, Is of 256neurons) and perturbed trajectories. The value approximates largest Lyapunov exponent of 1024 dimension dynamical system. We
confirmed that the system is stable at small c region (the gradient of divergence is 0 or below), and becomes chaotic at largerc (positive gradient of divergence). Stability may also be altered bya where smaller value leads to more stable dynamics, although further detailed analysis is needed.

The authors declare they have no conflicts of interest.

5. DISCUSSION
We constructed an autoassociative memory composed of 3-­variable
DSSN model which exhibit SFA. As a result, various dynamics wereACKNOWLEDGMENTS
This work was supported by JST SICORP Grant Number
JPMJSC15H1, Japan.

REFERENCES[1] E. M. Izhikevich, Which model to use for cortical spiking neurons?, IEEE Trans. Neural Netw. 15 (2004), 1063–1070.
[2] T. Kohno, K. Aihara, Digital spiking silicon neuron: concept andbehaviors in GJ-coupled network, Proceedings of International
Symposium on Artificial Life and Robotics, Beppu, 2007,
pp. OS3–OS6.

N. Takano and T. Kohno / Journal of Robotics, Networking and Artificial Life 7(1) 63–67[3] J. Li, Y. Katori, T. Kohno, An FPGA-based silicon neuronal network with selectable excitability silicon neurons, Front. Neurosci.
6 (2012), 183.
[4] A.C. Tang, A.M. Bartels, T.J. Sejnowski, Effects of cholinergic
modulation on responses of neocortical neurons to fluctuating
input, Cereb. Cortex 7 (1997), 502–509.
[5] J.P. Roach, E. Ben-Jacob, L.M. Sander, M.R. Zochowski,Formation and dynamics of waves in a cortical model of cholinergic modulation, PLoS Comput. Biol. 11 (2015), e1004449.
[6] J.P. Roach, L.M. Sander, M.R. Zochowski, Memory recall andspike-frequency adaptation, Phys. Rev. E 93 (2016), 052307.
[7] W. Kobayashi, T. Kohno, K. Aihara, 3-Variable digital spiking silicon neuron, Proceedings of the 24th Workshop on Circuit andSystems, Hyogo, 2011, pp. 1–5 (in Japanese).

67[8] T. Nanami, T. Kohno, Simple cortical and thalamic neuron models
for digital arithmetic circuit implementation, Front. Neurosci. 10(2016), 181.
[9] B. Ermentrout, Type I membranes, phase resetting curves andsynchrony, Neural Comput. 8 (1996), 979–1001.
[10] J.J. Hopfield, D.W. Tank, ‘Neural’ computation of decisions inoptimization problems, Biol. Cybern. 52 (1985), 141–152.
[11] H. Nozawa, Solution of the optimization problem using the neuralnet­work model as a globally coupled map, Phys. D 75 (1994),
179–189.

AUTHORS INTRODUCTION
Mr. Naruaki Takano
He is a graduate student at the Universityof Tokyo, Graduate School of Information
Science and Technology. He received hisB.S. in Information Science from Tokyo
Institute of Technology in 2018. His
research interests include computational
neuroscience and network modeling.

Dr. Takashi Kohno
He has been with the Institute of Industrial
Science at the University of Tokyo, Japan
since 2006, where he is currently aProfessor. He received the BE degree inmedicine in 1996 and the PhD degree inmathematical engineering in 2002 fromthe University of Tokyo, Japan.