1

Canonical Neural Models1
Frank Hoppensteadt1 and Eugene Izhikevich2
NeuronIntroduction
Mathematical modeling is a powerful tool in studying fundamental principles of information processing in the brain. Unfortunately, mathematical analysis of a certain neural model could be of limited
value since the results might depend on particularsof that model: Various models of the same neural
structure could produce different results. For example, if one obtains results studying a HodgkinHuxley-type model (see AXONAL MODELING) andthen augments the model by adding more parameters and variables to take into account more neurophysiological data, would similar results hold? A reasonable way to circumvent this problem is to derive
results that are largely independent of the model andthat can be observed in a class or a family of models.
Having understood the importance of consideringfamilies of neural models instead of a single model,
we carry out this task by reducing an entire familyof Hodgkin-Huxley-type models to a canonical model(for precise definitions see Sect. 4.1 in Hoppensteadt
and Izhikevich 1997). Briefly, a model is canonical fora family if there is a continuous change of variables
that transforms any other model from the family into
this one, as we illustrate in Figure 1. For example,
the entire family of weakly coupled oscillators of theform (1) can be converted into the canonical phasemodel (6), where Hij depend on the particulars of
the functions fi and gij . The change of variables
does not have to invertible, so the canonical modelis usually lower-dimensional, simple, and tractable.
Yet, it retains many important features of the family.
For example, if the canonical model has multiple attractors, then each member of the family has multiple
attractors.
The major advantage of considering canonicalmodels is that one can study universal neurocomputational properties that are shared by all members of the family since all such members can be put
into the canonical form by a continuous change ofvariables. Moreover, one need not actually present
such a change of variables explicitly, so derivation of
canonical models is possible even when the family isso broad that most of its members are given implicitly, e.g., in the abstract form (1). For example, thecanonical phase model (6) reveals universal computational abilities (e.g., oscillatory associative memory)that are shared by all oscillatory systems regardless

Family of Models
x=f1(x)x=f2(x)h1

x=f4(x)x=f3(x)h2

h3

y=g(y)

h4

Canonical
Model

Figure 1: Dynamical system ẏ = g(y), is a canonicalmodel for the family {f1 , f2 , f3 , f4 } of neural models ẋ = f (x) because each such model can be transformed into the form ẏ = g(y) by the continuouschange of variables hi .
of the nature of each oscillator or the particulars of
the equations that describe it. Thus, the canonicalmodel approach provides a rigorous way to obtain
results when only partial information about neuron
dynamics is known. Many examples are given below.
The process of deriving canonical neural models
is more an art than a science, since a general algorithm for doing this is not known. However, muchsuccess has been achieved when we consider weakly
connected networks of neurons whose activity is neara bifurcation, which often occurs when the membranepotential is near the threshold value, see DYNAMICS
AND BIFURCATION OF NEURAL NETWORKS
and PHASE PLANE ANALYSIS OF NEURAL ACTIVITY. We review such bifurcations and corresponding canonical models. Their rigorous derivation and detailed analysis can be found in the bookby Hoppensteadt and Izhikevich (1997).

Weakly Connected Neural Networks

The assumption of weak neuronal connections isbased on the observation that the typical size of apostsynaptic potential is less than 1 mV, which issmall in comparison with the mean size necessary todischarge a cell (around 20 mV) or the averaged sizeof the action potential (around 100 mV); see detailed
review of relevant electrophysiological data in Chap1 Invited chapter to Arbib MA (Ed) Brain Theory and Neural Networks. Second Edition. The MIT press, Cambridge, ter 1 in Hoppensteadt and Izhikevich (1997). FromMA, 2001
the mathematical point of view this results in neural2models of “weakly connected” form

and gij and the value of the parameters λi do notaffect the form of the canonical model, but only parnXẋi = f (xi , λi ) + ε
gij (xi , xj , ε) ,(1) ticulars of the parameters ri and sij . Thus, studying the canonical model (2) one can gain some inj=1
sight into neuro-computational behavior of any neuwhere each vector xi ∈ Rm describes membrane ral model near a cusp bifurcation, whether it is apotential, gating variables, and other electrophys- simple sigmoidal neuron or a biophysically detailed
iological variables of the i-th neuron (see ION conductance-based (Hodgkin-Huxley-type) neuron.
The canonical model (2) is quite simple: Each
CHANNELS: KEYS TO NEURONAL SPECIALIZATION). Each vector λi ∈ Rl denotes various bio- equation has only one non-linear term, namely, yi3 ,physical parameters of the neuron. The function f and two internal parameters, ri and sii . Still, thedescribes the neuron’s dynamics, and the functions Cohen-Grossberg-Hopfield convergence theorem apgij describe connections between the neurons. The plies, which means that the canonical model has thedimensionless parameter ε ¿ 1 is small, reflecting same neuro-computational properties as the standard Hopfield network (see COMPUTING WITH
the strength of connections between neurons.
ATTRACTORS).

Bistability And Hysteresis

Theorem 1 (Cohen-Grossberg-Hopfield ConBistable and hysteretic dynamics are ubiquitous in vergence Theorem) If the connection matrix S =neural models, and they may play important roles in (sij ) is symmetric, then the canonical neural network
biological neurons. The cusp bifurcation depicted in (2) is a gradient system.
One can easily check that

Depo

E(y) = −lariza

tion

x
λ

Hype
rp

olarization

Cusp Point

Figure 2: Cusp surface.
Figure 2 is one of the simplest bifurcations leading tosuch dynamics. For example, the sigmoidal neuronẋ = −x + aS(x) ,S(x) = 1/(1 + e−x ) ,is at a cusp bifurcation point x = 0.5 when a = 4. It
is bistable when a > 4. If each neuron in the weakly
connected network (1) is near a supercritical cusp bifurcation, then the entire network can be transformed
into the canonical form (Hoppensteadt and Izhikevich
1997)
n
X
yi0 = ri − yi3 +sij yj(2)
j=1

where each scalar yi ∈ R describes re-scaled dynamics of the ith neuron. Particulars of the functions fn
1 X
1
sij yi yj(ri yi − yi4 ) −4
2 i,j=1i=1nXis a potential function for (2) in the sense thatyi0 = −∂E/∂yi , see also ENERGY FUNCTIONSFOR NEURAL NETWORKS.

Small Amplitude Oscillations
Many biophysically detailed neural models can exhibit small-amplitude (damped) oscillations of themembrane potential, especially when the system isnear transition from rest state to periodic activity.
In the simplest case this corresponds to the supercritical Andronov-Hopf bifurcation (Figure 3). Many
weakly connected networks (1) of such neurons canbe transformed into the canonical modelzi0 = (ri + iωi )zi − zi |zi |2 +nX

cij zj(3)

j=1

by a continuous changeof variables (Aronson et al.
√1990). Here i = −1, and each complex variablezi ∈ C describes oscillatory activity of the ith neuron.
Again, particulars of the form of the functions f and
gij in (1) affect only the values of the parameters ri ,and ωi and the complex-valued synaptic coefficientscij ∈ C.
Even though the canonical model (3) exhibits oscillatory dynamics, one can still prove the following

3

x
λ

for (3), see ENERGY FUNCTIONS FOR NEURALNETWORKS.
The self-adjoint synaptic matrix arises naturally
when one considers complex Hebbian learning rules(Hoppensteadt and Izhikevich 1996)
k

cij =

1 X s ¯kξ ξn s=1 i j(5)where each vector ξ s = (ξ1s , . . . , ξns ) ∈ Cn denotesa pattern of phase relations between neurons to bexx
memorized, see also HEBBIAN SYNAPTIC PLASx
TICITY. Notice that the problem of negative (mirror) images does not arise in oscillatory neural networks, since both ξ k and −ξ k result in the same phaserelations.
The key difference between the Hopfield-Grossberg
network and the oscillatory network (3) is that memorized images correspond to equilibrium (point) atx
xx
tractors in the former and to limit cycle attractors intt
the latter. Pattern recognition by an oscillatory neutral network involves convergence to the corresponding limit cycle attractor, which results in synchroFigure 3: Supercritical Andronov-Hopf bifurcation in nization of the network activity with an appropriateẋ = f (x, λ). Left: The rest state is stable. Mid- phase relation between neurons, as in Fig. 4; see alsodle: The rest state is losing stability giving birth to a COMPUTING WITH ATTRACTORS.
stable limit cycle corresponding to periodic activity.
Right: The system exhibits periodic activity.
Large Amplitude Oscillations
analogue of the Cohen-Grossberg convergence theorem, which implies that the canonical model (3) has
oscillatory associative memory; that is, it can memorize and retrieve complex oscillatory patterns (Hoppensteadt and Izhikevich 1996); see Fig. 4.
Theorem 2 (Synchronization Theorem for Oscillatory Neural Networks) If in the canonicalneural network (3) all neurons have equal frequencies
ω1 = · · · = ωn and the connection matrix C = (cij )is self-adjoint, i.e.,

Suppose that neurons in the weakly connected network (1) exhibit periodic spiking; see Figure 5 andCHAINS OF COUPLED OSCILLATORS; COLLECTIVE BEHAVIOR OF COUPLED OSCILLATORS and PHASE-PLANE ANALYSIS OF NEURAL ACTIVITY. If they have nearly equal frequencies, then the network can be transformed into thephase canonical model
ϕ0i = ωi +n
X

Hij (ϕj − ϕi )

(6)

j=1

where each ϕi ∈ S1 is a one-dimensional (angle) variable that describes the phase of the ith oscillator
then the network always converges to an oscillatory along the limit cycle attractor corresponding to its
phase-locked pattern; that is, the neurons oscillate periodic spiking(see Figure 5), and each Hij is a funcwith equal frequencies and constant, but not neces- tion that depends on f and gij that can be explicitly
sarily identical, phases. There could be many such computed using Malkin’s Theorem (Theorem 9.2 inphase-locked patterns corresponding to many memo- Hoppensteadt and Izhikevich 1997).
The phase canonical model (6) describes frequency
rized images.
locking, phase locking, and synchronization properThe proof follows from the existence of an orbital ties of the original system (1). Therefore, to unenergy function
derstand these and other non-linear phenomena thatmight take place in oscillating neural networks, it
n
n
X
X
1
usually suffices to consider the phase model. In parcij z̄i zj
E(z) = −(ri |zi |2 − |zi |4 ) −2ticular, one can glimpse the universal computational
i,j=1
i=1
cij = c̄ji

for all i and j,

(4)

4

InitialState

Oscillatory Activity of the Network

12

60

Re z 10
Re z 2

InitialState

Relative Phase PatternFigure 4: Pattern recognition via phase locking by the oscillatory canonical model (3). Complex Hebbianlearning rule (5) was used to memorize patterns “1”, “2”, and “3”. When the distorted pattern “1” ispresented as an initial state, the neurons synchronize with the phase relations corresponding to the memorized
pattern “1”.

0.7

0.6

w

0.5

0.4

0.3

Morris-Lecar

0.2

0.1

0
0.5

V
0.4

0.3

0.2

0.1

0

0.1

0.2

0.3

0.4

for all i and j, then the canonical phase model (6)converges to a phase-locked pattern ϕi (t) → ω1 t + φifor all i, so the neurons oscillate with equal frequencies (ω1 ) and constant phase relations (ϕi (t)−ϕj (t) =V(t) φi − φj ). In this sense the network dynamic is synchronized. There could be many stable synchronizedpatterns corresponding to many memorized images.
The proof is based on the observation that the phasecanonical model (6) has the energy function

0.8

0.75

n

0.7

n

0.65

0.6

Hodgkin-Huxley

E(ϕ) =

0.55

0.5

V(t)

0.45

1 X
Rij (ϕj − ϕi )2 i,j=1

V

0.4

0
=
where Rij is the antiderivative of Hij ; that is, Rij
Hij , see Theorem 9.15 in Hoppensteadt and Izhikevich 1997, and ENERGY FUNCTIONS FOR NEUFigure 5: Examples of large amplitude limit cycle RAL NETWORKS.
attractors corresponding to periodic spiking in two
For example, Kuramoto’s model (1984)
biophysically detailed neural models (Morris-Lecarn
X(1981) and Hodgkin-Huxley (1952)).
ϕ0i = ωi +sij sin(ϕj + ψij − ϕi )
(8)

0.35
20

0

20

40

60

80

100

j=1

abilities that are shared by all oscillatory systems re- has such an oscillatory associative memory whengardless of the nature of each oscillator or the par- ω = · · · = ω1nticulars of the equations that describe it. Indeed, one
sij = sjiand
ψij = −ψji ,can prove the following analogue of Theorem 2
Theorem 3 (Synchronization Theorem for Oscillatory Neural Networks) If all oscillators in (6)have equal frequencies; i.e., ω1 = · · · = ωn , and theconnection functions Hij have pairwise odd form; i.e.,
Hij (−ψ) = −Hji (ψ)(7)

for all i and j. If we denote cij = sij eiψij , then these
conditions have the form (4). The energy function
for Kuramoto’s model isnE(ϕ) = −1 X
sij cos(ϕj + ψij − ϕi ) .
2 i,j=1

5

0.7

0.6

Limit Cycle

Action Potential(Spike)

w

0.5

0.4

0.3

0.2

0.1

Perturbation

V

0
0.5

0

Rest State

0.1

0.2

0.3

V(t)

Node

Saddle0.4

Threshold

Excitable

Figure 6: Neural excitability in Morris-Lecar (1981)
neuron having fast Ca2+ and slow K+ voltage-gatedcurrents. The rest state (black circle) is stable, but
small perturbations can push the voltage beyond thethreshold (white circle) thereby causing a large amplitude excursion – action potential. The voltage variable changes slowly near the rest states, but fast during the generation of action potentials.
There are various estimates of the storage capacityof the network, as discussed by Vicente et al. (1996).
In particular, those authors found a time scale during which oscillatory networks can have better performance than Cohen-Grossberg-Hopfield-type attractor neural networks.
Since neither the form of the functions f and gij ,nor the dimension of each oscillator in (1) were specified, one could take the above result to the extreme
and claim that anything that can oscillate can also be
used for computing, as for associative pattern recognition, etc. The only problem is how to couple theoscillators so that (7) is satisfied.

Neural Excitability
An interesting intermediate case between rest and periodic spiking behavior is when a neural system is excitable; that is, it is at rest, but can generate a largeamplitude spike in response to a small perturbation;
see Figure 6 and PHASE-PLANE ANALYSIS OFNEURAL ACTIVITY and OSCILLATORY AND
BURSTING PROPERTIES OF NEURONS. A simple but useful criterion for classifying neural excitability was suggested by Hodgkin (1948), who stimulated
cells by applying currents of various strengths. Whenthe current is weak the cell is quiet. When the current
is strong enough, the cell starts to fire repeatedly. He
suggested the following classification according to theemerging frequency of firing (see Figure 7)

Saddle-Node on Invariant
Circle Bifurcation

Repetitive Spiking

Figure 8: Class 1 neural excitability via saddle-node
on invariant circle bifurcation: The threshold state(saddle) approaches the rest state (node), they coalesce and annihilate each other leaving only limit
cycle attractor. The oscillation on the attractor has
two time scales: slow transition through the “ghost”
of the saddle-node bifurcation and fast rotation along
the rest of the limit cycle.
• Class 2 neural excitability. Action potentials
are generated in a certain frequency band that is
relatively insensitive to changes in the strengthof the applied current.
Their class of excitability influences neurocomputational properties of cells (see review by
Izhikevich 2000).
For example, Class 1 neuralsystems have a well-defined threshold manifold fortheir state variables, beyond which they generate alarge amplitude spike. They generate an all-or-none
response, and they act as integrators, meaning thatthe higher the frequency of the incoming pulses,
the sooner they fire. In contrast, Class 2 neuralsystems do not have a threshold manifold. They cangenerate spikes of arbitrary intermediate amplitude,
and they act as resonators. That is, they respond tocertain resonant frequencies of the incoming pulses.
Increasing the incoming frequency may delay or eventerminate their response.
A canonical model for Class 1 excitable systems is
described below, while the canonical model for Class
2 systems has yet to be found.

Class 1 Excitable Systems

Class 1 excitable systems are understood relatively
well (Rinzel and Ermentrout 1989, Ermentrout 1996,
Hoppensteadt and Izhikevich 1997, Izhikevich 2000).
The transition from rest to periodic spiking in such
systems occurs via a saddle-node on invariant circle bifurcation, as we depict in Figure 8, see also
• Class 1 neural excitability. Action potentials DYNAMICS AND BIFURCATION OF NEURAL
can be generated with arbitrarily low frequency. NETWORKS and OSCILLATORY AND BURSTThe frequency increases with increasing the ap- ING PROPERTIES OF NEURONS. A weakly connected network of such neurons can be transformed
plied current.

6Morris-Lecar

Hodgkin-Huxley

0.2

0

V(t)

20

V(t)

0

I

0

0.9

100

200

300

400

500

600

700

800

900

1000

0

I

17.5

0.6

200

300

400

500

600

700

800

900

1000

I

Spiking
Frequency

7.5

Spiking
Frequency

V(t)

100

I

I

Class 2 Neural Excitability

Class 1 Neural Excitability

Figure 7: Transition from rest to repetitive spiking in two biophysical models when the strength of appliedcurrent, I, increases. The neural excitability is classified according to the frequency of emerging spiking.
into a canonical model, which can be approximated
by


n
X
ϑ0i = 1 − cos ϑi + (1 + cos ϑi ) ri +sij δ(ϑj − π)

Excited(Regenerative)

Spike

j=1

(9)
where ϑi ∈ S1 is the phase of the ith neuron along
the limit cycle corresponding to the spiking solution.
Again, particulars of the functions f and gij in (1)do not affect the form of the canonical model above,
but only affect the values of the parameters ri and sij ,which can be computed explicitly (Hoppensteadt andIzhikevich 1997, Chapter 8). Notice that the canonical model (9) is pulse-coupled, whereas the original
weakly coupled network (1) is not. The qualitative
reason for pulse coupling is that the voltage changes
are extremely slow most of the time because of theproximity to the rest state, but they are relatively instantaneous during the generation of an action potential. Hence the duration of coupling looks infinitesimal on the slow time scale.
The neuron is quiescent when ri < 0 (Fig. 8, left)
and fires periodically when ri > 0 (Fig. 8, right).
It fires a spike exactly when ϑi crosses the value π,which results in a step-like increase in the phases ofother neurons. Hence, the canonical model (9) is apulse coupled neural network (Izhikevich 1999). It
has many important physiological features including
absolute and relative refractory periods; see Figure 9.
Indeed, the effect of every incoming pulse depends on
the internal state of the neuron since it is multipliedby the function (1 + cos ϑi ). The effect is maximalπ

Threshold
Potential
10S

Absolute
Refractory

Rest
Potential

Relative Refractory

Figure 9: Diagram of the canonical model (9) for
class 1 neural excitability (From Hoppensteadt andIzhikevich 1997).

when the neuron is near rest since (1 + cos ϑi ) ≈ 2
when ϑi ≈ 0. It is minimal when the neuron is generating a spike since (1 + cos ϑi ) ≈ 0 when ϑi ≈ π.
A canonical model for slowly connected Class 1 excitable neurons with spike frequency adaptation has
the form (Izhikevich 2000)

ϑ0i

=

1 − cos ϑi + (1 + cos ϑi ) ri +wi0=

δ(ϑi − π) − ηwinXsij wj 

j=1

where wi describes slow synaptic processes. The termsii wi denotes not a self-synapse, but a slow spike frequency adaptation (sii < 0) or facilitation (sii > 0)process.

7

Discussion
The canonical model approach to computational neuroscience provides a rigorous way to derive simple yet
accurate models that describe single cell or network
dynamics (see also SINGLE-CELL MODELS). Sucha derivation is possible even when no assumptions
are made regarding the detailed form of equations
describing neural activity. Indeed, we specify neither f nor gij in (1). The only assumptions we makeare those concerning the dynamics of each neuron –whether it is quiescent, excitable, periodic spiking,etc. Nevertheless, any such neural system can betransformed into a canonical model by a piece-wise
continuous change of variables.
Derivation of canonical models can be a daunting
mathematical task. However, once found, the canonical models provide invaluable information about universal neuro-computational properties shared by alarge family of neural systems. For example, studying the canonical model (9) sheds light on behavior
of all Class 1 excitable systems and their networks
regardless of the details of equations describing theirdynamics.
Road Map: BIOLOGICAL NEURONS AND NETWORKS; COOPERATIVE PHENOMENA; DYNAMIC SYSTEMS AND OPTIMIZATION
Background: DYNAMICS AND BIFURCATION
OF NEURAL NETWORKS; PHASE PLANEANALYSIS OF NEURAL ACTIVITY
Related Reading:
CHAINS OF COUPLEDOSCILLATORS; COLLECTIVE BEHAVIOR OF
COUPLED OSCILLATORS; COMPUTING WITHATTRACTORS; COOPERATIVE PHENOMENA;ENERGY FUNCTIONS FOR NEURAL NETWORKS; PATTERN FORMATION, BIOLOGICAL

References
Aronson D.G., Ermentrout G.B., and Kopell N.,
1990, Amplitude response of coupled oscillators.
Physica D, 41:403–449.
Ermentrout G.B., 1996, Type I Membranes, Phase
Resetting Curves, and Synchrony. Neural Computation, 8:979–1001.
Hodgkin A.L., 1948, The local electric changes associated with repetitive action in a non-medulated
axon. Journal of Physiology, 107:165–181.
Hodgkin A.L. and Huxley A.F., 1952, A quantitative description of membrane current and application to conduction and excitation in nerve. Journalof Physiology, 117:500–544.

Hoppensteadt FC and Izhikevich E.M., 1997,
Weakly Connected Neural Networks, Springer-Verlag,
NY
Hoppensteadt F.C. and Izhikevich E.M., 1996,
Synaptic organizations and dynamical properties of
weakly connected neural oscillators. II. Learning ofphase information. Biological Cybernetics, 75:129–
135.
Izhikevich E.M., 1999, Class 1 Neural Excitability, Conventional Synapses, Weakly Connected Networks, and Mathematical Foundations of PulseCoupled Models, IEEE transactions on neural networks, 10:499–507
Izhikevich E.M., 2000, Neural Excitability, Spiking, and Bursting, International Journal of Bifurcation and Chaos vol.10, n.3.
Kuramoto Y., 1984, Chemical Oscillations, Waves,and Turbulence. Springer-Verlag, New York.
Morris C. and Lecar H., 1981, Voltage Oscillations in the Barnacle Giant Muscle Fiber, BiophysicalJournal, 35:193–213.
Rinzel J. and Ermentrout G.B., 1989, Analysis of
Neural Excitability and Oscillations. In Koch C.,
Segev I. (eds) Methods in Neuronal Modeling, The
MIT Press, Cambridge, Mass.
Vicente C.J., Arenas A., and Bonilla L. L. (1996)
On the short-term dynamics of networks of Hebbian
coupled oscillators, J. Phys. A, L9-L16.