Learning Shapes Bifurcations of Neural
Dynamics upon External Stimuli
Tomoki Kurikawa, Kunihiko Kaneko
University of Tokyo, Depertment of Basic Science
Komaba 3-8-1, Meguro-ku, Tokyo, Japankurikawa@complex.c.u-tokyo.ac.jp

Abstract. Memory is often considered to be embedded into one of theattractors in neural dynamical systems, which provides an appropriate
output depending on the initial state specified by an input. However,memory is recalled only under the presence of external inputs. Without
such inputs, neural states do not provide such memorized outputs. Hence,each of memories do not necessarily correspond to an attractor of the
dynamical system without input and do correspond to an attractor ofthe dynamics system with input. With this background, we propose thatmemory recall occurs when the neural activity changes to an appropriate
output activity upon the application of an input. We introduce a neuralnetwork model that enables learning of such memories. After the learning
process is complete, the neural dynamics is shaped so that it changes tothe desired target with each input. This change is analyzed as bifurcation
in a dynamical system. Conditions on timescales for synaptic plasticity
are obtained to achieve the maximal memory capacity.
Keywords: neural network, bifurcation, associative reward-penalty

1

introduction

One of the most important features of the brain is the ability to learn andgenerate an appropriate response to external stimuli. Output responses to the
input stimuli are memorized as a result of synaptic plasticity. A wide varietyof neural network models have been proposed to study how synaptic strength
pattern is formed to memorize given input-output (I/O) relationships. In mostof the studies[1], inputs are supplied as the initial conditions for neural activity, whose temporal evolution results in the generation of the desired outputs.
However, recent experimental studies have shown that there exist structured
spontaneous neural activity even in the absence of external stimuli[2]. Upon external stimuli, such neural activity is modified to provide appropriate response
neural activities[3]. Considering these studies, we propose a novel perspectiveof the memorization of I/O relationships: If a neural system memorizes an I/O
relationship, the spontaneous dynamics is modulated by a given input to provide
a required output. In other words, the input changes the flow structure of theneural dynamical system to generate the corresponding output.

2

On the basis of this idea, we address the following questions: Can we construct
an appropriate neural network model to demonstrate the learning process under
biologically plausible assumptions? If so, under what conditions would learning
be possible? What changes in the neural activity brings about the output when
an input is applicated ?

2

Model

In modeling learning, we postulate the following two conditions that satisfy
the biological requirements of the brain: (i) The learning process should not require detailed error information. In other words, the number of error information
should be considerably smaller than the number of neurons. For example, the error back-propagation algorithm requires error information corresponding to eachof the output neurons. In the case of biological learning with neural network,
however, it is difficult to transmit such large amounts of information specificallyto each neuron. (ii) I/O relationships should be learned one by one, i.e., thenext novel I/O relationship is learned after one relationship has been learned
with preserving the previously learned relationships. In contrast, in most neuralnetwork algorithms, many relationships are simultaneously learned by graduallychanging the synaptic strength until all the relationships are memorized.
We introduce a layered network model consisting of input, hidden, and output layers along with a reinforcement learning such algorithm as the associative reward-penalty (ARP)(Fig.1A)[4], so that this model satisfy the abovementioned conditions. In this model, several I/O relationships are learned one
by one with only a single error signal which is given as the distance between theactivity pattern of the output neurons and a prescribed target pattern. During
the learning process, the synaptic strength varies in accordance with the Hebbian and anti-Hebbian rules, switched depending on the magnitude of the error
signal.
To be specific, we adopt the following model with N neurons in each layer.
Three types of synapses are considered: forward synapses (FSs), backward synapses(BSs), and mutually inhibiting intralayer synapses (ISs). FSs connect the neurons in the input layer to those in the hidden layer and the neurons in the hidden
layer to those in the output layer. BSs connect the neurons in the output layer
to those in the hidden layer, while ISs connect the neurons within a given layer(hidden layer or output layer).
The neural activity in the input layer is determined by an input pattern I, avector whose element takes the value 0 or 1; the magnitude of the vector (input
strength) is η (Eq.1). The neural activities in the other layers change, as shownby the rate coding model (Eq.2):

xi = ηIi
τNA(I ∈ {0, 1})ẋi = 1/(1 + exp(−βui + θ)) − xi

(input layer)(1)(the other layers)

(2)

3

1 0

10

er

ay

e
dd

n

pu
ut
O

la

a
tl

ge

ye

ye

r

１
Reward signal

Hi

tl

Cell index

puIni) Raster plot

0

r

0 1

0

0 0

tξ

1

Target 1ii) Error

1st set2nd set

0.3
0.15
0
0

r
Ta

Target 2400

8001200 1600

Time
FS

BS

IS

A) Network architectureB) Model dynamics

Fig. 1. A) Schematic representation of the network architecture of our model. B) Dynamics of neural activities during the learning of two input-output (I/O) relationships. I/O relationships are learned in the search phase by the anti-Hebbian rule(0 < t < 350,800 < t < 1200) and in the stabilization phase by the Hebbian rule(350 < t < 800,1200 < t < 1700). As initial conditions for the network, we set τ NA = 1,
τ BS = 8, and τ FS = 64 and assign the synaptic strength a random value between 0 and
1, except in the case of the ISs. (i) Raster plot of neurons in the output layer. Red bar
represents the high activity of each neuron (xi > 0.9). (ii) Time series of amplitude ofthe error signal E between the output and the target patterns. Color bar above thetime series represents each set of (I/O) relationships.

where xi is the firing rate of a neuron i, and ui is thecurrentPapplied to each
PNinput
N
FS in
BS outneuron i. The input current is given by uhid=J+i
j=1 ij xj +j=1 Jij xjPPNFS hidIS hidout
J xj for the neurons in the hidden layer and ui = j=1 Jij xj +Pj6=i IS out
FSBSJ
xj for the neurons in the output layer. Here, Jij(Jij) is the strength
j6=iof the forward (backward) synapse from a presynaptic neuron j to a postsynaptic neuron i. J IS is the strength parameter for the mutually inhibiting IS; thisparameter assumes a fixed and identical value for all synapses. The parameters
are set at τ NA = 1, β = 42, θ = 2.5, η = 1.0, J IS = −1.0, and N = 10.
For each input pattern, we prescribe a target pattern ξ as an N -dimensional
vector whose element takes the value 0 or 1. Sparse input and target patterns, inwhich only one neuron is activated, are chosen. By describing the neural activity
in the output layer as the N -dimensional vector X out , the learning task moves
the error E = |X out − ξ|2 /N closer to zero, as measured by the Euclidean norm.
Synaptic plasticity is necessary for achieving the learning in a neural network. For simplicity, we maintain the strength of the ISs constant and vary thestrengths of the FSs and BSs. In accordance with the Hebb scheme and ARP[4],we assume that the synaptic dynamics depends on the activities of the pre- and4postsynaptic neurons as well as on the reward signal R determined by the error
signal E, asp= Rp (xi − r)xj
τ p J˙ij(J ≥ 0)(p = FS or BS)(3)Here, r is the spontaneous firing rate(r is set at 0.1). The synaptic plasticity inthis model has two characteristic features as follows.
(i) Plasticity switched by the error:
As mentioned earlier, the sign of R determines the synaptic plasticity. The
sign of R changes with the magnitude of E such that

R

FS

=



1 for
−1 for

E≤ǫ
E>ǫ

R

BS

=



0 for E ≤ ǫ−1 for E > ǫ(4)

When the output pattern is close to (distance from) the target pattern, i.e.,E ≤ (>)ǫ, the synaptic plasticity follows the Hebbian (anti-Hebbian) rule, as
derived by substituting Eq.4 to Eq.3. The Hebbian (anti-Hebbian) rule stabilizes(destabilizes) the ongoing neural activity. In this manner, the error switches thesynaptic plasticity between the Hebbian and anti-Hebbian rules. Note that under
Hebbian rule, only the strength of the FS varies so that memories of the I/O
relationships are embedded in the FSs.
(ii) Multiple time scales:
In most neural network studies, only two time scales are considered: One forneural activities and the other for the synaptic plasticity. Considering varietyof timescales of the synaptic plasticity, we introduce two time scales for theplasticity of the FSs and BSs, τ FS and τ BS in Eq.3, respectively. As will be shown
later, I/O relations are successfully memorized when the difference between thetime scales is appropriate.

3

Results and Analysis

3.1

Learning Process

We first show that this model can be used to learn I/O relationships; the neuralactivity is varied when searching the target and stabilized when matching thetarget(Fig. 1B). When the error is large, the present neural activity becomes unstable by the anti-Hebbian rule and hence the neural dynamics itinerate among
different patterns (0 < t < 350). The target pattern is searched during this itineration. We term this period the search phase in what follows. At t ∼ 350, themagnitude of the error reduces to a sufficient extent when the output dynamics
are within the neighborhood of ǫ. Once this occurs, the neural activity is stabilized as per the Hebbian rule, and the output activity remains close to the target(350 < t < 800). Thus, the activated synapses are continuously strengthened until a new target is presented. This period is called the stabilization phase. Att ∼ 800, we switch a new input and the corresponding target pair to learn this.

5

At that time, the distance between the output pattern and the target patternincreases again, and therefore, the searching process progresses on the basis of
the anti-Hebbian rule (800 < t < 1200). In this manner, the neural activity
can reach the target by switching between anti-Hebbian and Hebbian rules alternately for synaptic plasticity, depending on the error. Now, we successively
provide new input-target pairs after the time interval of stabilization phase,
which is sufficiently long for learning an I/O relationship.
1.0

ξ2

ξ2

ξ

0

ξ4

3

1.0

1.0
3

ξ 4

ξ

0 0

(i) early stage in learning process
ξ4

Total # of attractors# of fixed points

6

ξ2

ξ3

0

0

0

ξ3 +

ξ 6×

5

0.3

1.0 1.0

2

ξ

+ξ

×0

.3

0
1

(ii) late stage in learning process
Trajectory Target ξi1Trajectory Target ξ

without input

with input 3

with input 2

with input 4

3

5

7

9

# of limit cycles

ξ5

1.0

7

0

Index of the targetiA) Temporal evolutions in several condtionsB) Change in number of attractors

Fig. 2. Analysis of changes in the flow structure of the spontaneous neural activity
during the learning process. A)Temporal evolution of neural activity in the output layer.
Neural activity in the output layer is projected from the N -dimensional space consistingof neural activities in the output layer to three-dimensional space by obtaining the
product of the output activity and the target pattern(s). (i) Neural dynamics after
learning four I/O relationships. Each axis represents the product of neural activity
and the corresponding target pattern. (ii) Neural dynamics after learning seven I/O
relationships. Each axis represents the product of the neural activity and corresponding
combined target patterns. B) Change in the number of attractors during the learning
process. Numbers of fixed-point attractors (green line) and limit-cycle attractors (blueline) and the total number of attractors (red line) in the absence of inputs as a functionof the number of learning steps, i.e., number of learned targets, are plotted.

3.2

Memories as Bifurcations

The learning process changes the flow structure in neural dynamical system bothin the presence and the absence of external inputs. We examine the typical orbitsof neural activity without any input and changes of such orbits with inputs in a
dynamical system with a fixed synaptic strength after each learning step. Figure

6

2 shows examples of typical orbit in the attractor, as shown by the results oflearning four and seven I/O relations. After four I/O relations are learned (Fig.
2A(i)), the neural activity in the output layer in the absence of any input isitinerant over three patterns that are close to three of the target patterns until
the neural activity converges to a fixed point. Figure 2A(ii) shows an example
of a limit-cycle attractor in the absence of inputs after seven I/O relations arelearned, such that the orbit in the attractor (not a transient orbit as shown in
Fig. 2A(i)) itinerates over the targets in the cyclic order 2, 6, 3, 4, 5. In bothcases, these target patterns do not exist as fixed-point attractors without inputs.
In application of the input, the fixed point and/or the limit cycle collapse and thecorresponding target pattern change to a stable fixed-point attractor. Hence, thisnetwork memorizes these I/O relations as bifurcations. The number of memories
varies through the learning process. Generally, as the learning progresses further,
the number of memories increases in adequate time scales and that of fixed-point
attractors begins to decrease, as these attractors are replaced by one or more
limit-cycle attractors (Fig. 2B).
3.3

Memory Capacity: Dependence on Time Scales for Synaptic
Plasticity

In this manner, our model can memorize I/O relationships successively up tosome limit. The maximal number of memorized I/O relationships through learning process gives a memory capacity. This capacity depends on three time scales:τ NA for changes in the neural activity, τ BS and τ FS for the plasticity of the BSs
and FSs, respectively. From extensive numerical simulations, we found that the
memory capacity reaches the maximal possible number N under the sparse coding case, when the condition τ NA ≪ τ BS ≪ τ FS is satisfied. This implies thatwith a single synaptic time scale (τ FS = τ BS ) as in the case of usual learningmodels, the capacity is not high.
To explain this time scale relationship, we study the synaptic dynamics during the search phase. In the phase, the output activity may come close to one ofthe previously learned target patterns. Since this pattern differs from the current
target pattern, the attraction to it may be destroyed by the anti-Hebbian rule.
In general, the longer the output pattern stays close to a state corresponding
to a previously learned target pattern, the stronger is the destabilization of thestate. Hence, the degree of destabilization of the previous memory increases withthe residence time of the pattern on the corresponding state. As shown in Fig.
3, the residence time as a function of τ BS decreases to a minimum at the τ BSof the maximum memory capacity and increases as τ BS approaches τ NA or τ FS .
This can explain the dependence of the memory capacity on τ BS .
Now we discuss the origin of this behavior of residence time. First, τ FS determines the time scale of the memory decay, because the memory information
is embedded in the FSs (Eq.4). Second, a smaller value of either τ FS or τ BS determines the time scale of the search phase, because the search for the target ispossible only on the basis of the change in the flow structure by the anti-Hebbian

7
2600

2000
6
1400

ȫNA

2
1

ȫFS
10

Residence time

Memory capacity

10

800

100

ȫBS

Fig. 3. Memory capacity (green) and residence time (red) as a function of the time scaleof backward synapse τ BS . The other two time scales τ NA and τ FS are fixed at τ NA = 1
and τ FS = 64. See text for the definition of capacity and residence time. Computedfrom the average over 100 learning processes for each τ BS . Error bars indicate standard
deviations.

rule. Since the search phase should be sufficiently smaller than the memory decay time, τ BS ≪ τ FS is required to preserve the previous memory during thesearch phase. Third, τ NA determines the time scale for the neural dynamics under given fixed flow structure. If this value were larger than or of the same orderof τ BS , the phase structure would be modified before the neural activity change,
so that the approach to the target pattern would be hindered. Hence τ NA ≪ τ BS
is required. Accordingly the relationship τ NA ≪ τ BS ≪ τ FS is required for theeffective search for a new target without destroying the previous memory.

4

Discussion

In the present paper, we have proposed a novel dynamical systems model for the
memory, in which the learning process shapes the “appropriate” flow structureof spontaneous neural dynamics, through successive presentations of inputs andthe corresponding outputs. Memory recall is achieved as a result of the bifurcation of neural dynamics from spontaneous activity to an attractor that matches
the target pattern induced by the external input. This bifurcation viewpoint is
supported by, for example, a recent experimental study[3], in which the neuraldynamics of the olfactory system of insects were studied in the presence andabsence of odor stimuli. Here, we discuss how two above-mentioned features inour model can be implemented in out brain.
(i) Plasticity switched by the error:
In our brain, there exist several neural modulators such as dopamine, serotonin, norepinephrine, and acetylcholine. In particular, dopamine modulates thesynaptic plasticity at the hetero-synaptic[5] and is projected to the cerebral cortex broadly. These natures correspond to the synaptic dynamics determined by

8

the product between the reward, the activities of pre- and postsynaptic neurons and that regulated by the error signal globally. In addition, the activityof dopamine neuron producting dopamine is related to evaluation how consistent the response is with the request[6]. Hence, in our brain, the switch between
positive and negative plasticity corresponding to that between Hebbian and antiHebbian rules can be regulated by value evaluation through the concentrationof dopamine.
(ii) Multiple time scales:
Because the synaptic plasticity is propotional to the product between the
reward, the activities of pre- and postsynaptic neurons, the difference in thesensitivity of the reward signal or the activities of pre- and postsynaptic neurons
controls that in the time scale of the synaptic plasticity effectively. In our brain,
this sensitivity can be interpreted as the number and/or the type of receptors ofneural modulators and neurotransmitters.
To achieve the maximum number of memorized patterns, a proper relationship has to be satisfied among the time scales of the changes in the neural activity
and of the plasticity of the FSs and the BSs. By above discussion, the requested
relationships for successful learning can be implemented by the distribution ofneurons with adequate numbers and/or type of receptors in our brain.
According to our idea “memories as bifurcations”, the neural dynamics inthe presence and absence of different inputs are distinct and separated becauseof bifurcation, which stabilizes distinct memorized patterns. In contrast to theview “memories as attractors”, our interpretation allows for the coherent discussion of diverse dynamics depending on the applied inputs. In fact, spontaneous
neural activity is recently reported, which shows itinerancy over several states
tat are stabilized by inputs[7]. This gives remarkable agreement with our results.
This bifurcation against the input strength will be experimentally confirmed bymeasuring the neural activity depending on the external stimuli.

References
1. Hopfield JJ (1982) Neural networks and physical systems with emergent collective
computational abilities. Proc Natl Acad Sci USA 79:2554-2558.
2. Luczak A, Bartho P, Marguet SL, Buzsaki G, Harris KD (2007) Sequential structureof neocortical spontaneous activity in vivo. Proc Natl Acad Sci USA 104:347-352.
3. Mazor Ofer, Gilles Laurent (2005) Transient Dynamics versus Fixed Points in Odor
Representations by Locust Antennal Lobe Projection Neurons. Neuron 48:661-673.
4. Barto AG, Sutton RS, Brouwer PS (1981) Associative Search Network. Biol. Cybern.
40:201-211.
5. Jay TM (2003) Dopamine: a potential substrate for synaptic plasticity and memory
mechanism. Prog Neurobiol 69:375-390.
6. John N. J. Reynolds, Brian I. Hyland, Jeffery R. Wickens (2001) A cellular mechanism of reward-related learning. Nature 413:67-70.
7. Kenet T, Bibitchkov D, Tsodyks M, Grinvald A, Arieli A (2003) Spontaneously
emerging cortical representations of visual attributes. Nature 425:954-956