2011 International Symposium on Nonlinear Theory and its Applications
NOLTA2011, Kobe, Japan, September 4-7, 2011

Multi-Layer Perceptron with Pulse Glial Chain
Chihiro Ikuta, Yoko Uwate and Yoshifumi Nishio
Dept. of Electrical and Electronic Engineering, Tokushima University
2–1 Minami–Josanjima, Tokushima, Japan
Email: {ikuta, uwate, nishio}@ee.tokushima-u.ac.jp
Abstract—A glia is a nervous cell in the brain. Currently, the glia is known as a important cell for the human’scerebration. Because the glia transmits signals to neurons
and other glias. We notice features of the glia and considerto apply it for an artificial neural network.
In this paper, we propose a Multi-layer perceptron(MLP) with pulse glial chain. The pulse glial chain is inspired from the features of the glia. The glia generates apulse by a connecting neuron and it excites neighborhood
neurons and glias. The pulse is generated as diﬀerent glia,
thus, these pulses look like propagating into the network.
We show the MLP with pulse glial chain has better performance than the conventional MLP by computer simulations.

reaction. We confirm by computer simulations that the glial
chain improves the learning performance of the MLP byconnecting the neurons more eﬀectively than the conventional networks.
2. MLP with pulse glial chain
The MLP is one of the feed forward neural network andis composed by some neuron layers. The MLP is learned
by BP algorithm from the network error. This network cansolve many kinds of tasks, for example, pattern recognition,
pattern classification, data mining and so on. In this study,
our MLP has three layers (which is composed by 4–10–1) and hidden layer neurons have the glial chain shown inFig. 1.

1. Introduction
A glia is one of the nervous cell which is existing in abrain. The glia was known to supporting cell for neuron
works by carrying a nutrient for a long time. Recently,
some researchers discovered that this cell had high calcium
response capability, and that the calcium response was provoked by the glutamic acid [1][2]. The glutamic acid is an
important ion in the brain, because this ion is one of a neurotransmitter as a synapse. Thus, the glia is known to be
actively involved with information processing of the brain.
We have noticed the biological glia works in the brainand considered that apply this work to the artificial neural network. We used a Multi-Layer Perceptron (MLP) for
one of the applying model [3]. The MLP is the most famous feed forward neural network. It is known to solve
the pattern recognition, pattern classification, data mining
and so on. The MLP is composed by some neurons, andthe neurons compose some layer. These neurons connect
neighborhood layer’s neurons. The MLP is learned by
Back Propagation (BP) [4]. The network become to solve
linearly-inseparable problem, however, the BP algorithm
often falls into local minimum. Because, the BP uses thesteepest decent method. If the MLP can escape out fromthe local minimum, this network find a better solution.
In this study, we propose a MLP with pulse glial chain.
A pattern of glia firing looks like a pulse. The glia is firedby connecting neuron output. If the one glia is fired, aroundglias are fired chain reaction. In our proposed glial chain,
the glia generates the pulse output by the connecting neuron’s huge output, after that the other glia is excited chain

Figure 1: MLP with pulse glial chain.

2.1. Neuron updating rule
The neuron has multi-inputs and one output. We canprocess the neuron output by the deciding weight parameter between neurons. The standard neuron updating rule isdescribed by Eq. (1).
 n
∑

(1)
yi (t + 1) = f  wi j (t)x j (t) − θi (t) ,j=1

where y is a neuron output, x is an input from forward layer
neurons, w is a weight parameter, θ is a threshold and f is- 435 -an output function. The weight parameter and the threshold
are learned by BP algorithm.
The updating rule of the hidden layer’s neurons of theproposed neural network with the glial chain is modified asEq. (2).
 n∑
yi (t + 1) = f  wi j (t)x j (t) − θi (t) + αψi (t) ,(2)j=1where ψ : output of the glias, α : weight of glia outputs.
We use the sigmoid function for the output f as Eq. (3).
f (a) =1
1 + e−aand if the glia is excited, it can not excite during period ofinactivity.
Next, we show the pulse outputs when the glias do nothave the relationship of each other for in Fig. 3. Each pulse
detail is the same as the pulse of Fig. 2. However, in thesepulse outputs, they are independent from other glias, thus
the pulse outputs are generated at diﬀerent times as eachposition of glia. We confirm that the relationship of theglias is important for the MLP learning by comparing theMLPs which are connected glias of two diﬀerent conditionsof the generating pulse outputs.

(3)
0.5

2.2. Glial network

0.4

Currently, the glia is attract researchers attentions in thebiological or the medical fields. Because, the glia has important work for brain system. The glia has many kindsof ion receptors. Especially, the glia change Ca2+ density
in the brain and this calcium response likes pulse response[5][6]. The Ca2+ aﬀect to the neuron membrane potential.
Moreover, the glia’s calcium response occur chain reaction
as around glias [7][8].
In this study, we propose the pulse glial chain which isinspired from these features of glia. The pulse glial chain
is generated by the glias. One glia generates pulse output
and the pulse is propagating in the glial chain. We definethat only one glia (first glia as the glial chain) is excitedby the neuron output, after that other glia is excited by thisglia’s pulse output. Thereby, the glia’s pulse output looks
like propagating other glia passing into the glial chain. Thepulse output of first glia is defined in Eq. (4).

 1, (θn < y1 ) ∩ (θg > ψ1 (t))(4)
ψ1 (t + 1) =  γψ1 (t), else,

0.3
0.20.109753111121314151

61718191

141
121 131101 111

Figure 2: Propagating pulse by the pulse glial chain (D =1)

0.5
0.4
0.3
0.2
0.1
097

where ψ is the glia pulse output, γ is an attenuated parameter, θn is a threshold of the exciting glia and θg is a threshold of a glia’s refractory period. Other glias have diﬀerent conditions of exciting. These glias are excited by firstglia output. If the first glia is excited by neuron output,
this information is propagating into the glial chain. After
that, other glias excite and generate the pulse output. Theseglias’ exciting conditions are defined in Eq. (5).

 1, (θt < ψ1 (t − i ∗ D)) ∩ (θg > ψi (t))ψi (t + 1) = (5) γψi (t), else,
where D is number of delay and θt is a threshold of first gliaoutput. Figure 2 is an example of the glia generating pulse.
In this figure, the pulse conditions are γ = 0.8, θn = 0.845 ,θn < y1 (every time) and D = 1. The first glia generates
pulses by neuron output. The pulses excite neighborhood
glia, thereby, the pulse looks like propagating into the glialchain. One glia output is decreased at an exponential rate

5311

1121

31

41

51

61

71

8191

141
121 131101 111

Figure 3: Generating the pulse at random timing.

3. Simulations
In this section, we confirm the eﬃciency of the glial
chain to the learning performance of the MLPs. We compare the four kinds of MLPs, which are(1) Conventional MLP(2) MLP with pulse glial chain (D = 1)
(3) MLP with pulse glial chain (D = 0)
(4) MLP with random timing pulse.

- 436 -3.1. Simulation task
In this simulation, the MLP learns classification of twoskew tent maps. The skew tent map generates the chaotic
time series and the MLP learns time change. The skew tent
map is described by Eq. (6).
 2ϕ(t)+1−A (−1 ≤ ϕ(t) ≤ A)
1+Aϕi (t + 1) = ,
 −2ϕ(t)+1+A(A<ϕ(t)≤1)
1−A(6)

where ϕ is a chaotic oscillation as each time and A isfeature of the skew tent map. If A is changed, the skew tent
map has diﬀerent oscillations. In the skew tent map, thechaotic time series change between −1 and 1. The neuron
inputs value are between 0 and 1. Thus, we normalize thechaotic time series by skew tent map to between 0 and 1.
Figure 4 is a map of the skew tent map as the two diﬀerentA.

50000 times, after that we check the each index. We obtainthe statistic results from 200 trials. From this result, we cansee that the MLP with the pulse glial chain is the best resultof all as every index. In the conventional MLP, the learning
performance is the worst, because this network often falls
into local minimum during learning. The average of errorof the MLP with pulse glial chain (D = 0) is worse than theMLP with pulse glial chain (D = 1). We consider this reason that this MLP (D = 0) is given too much energy fromthe pulse glial chain. Because the neurons in this MLP, aregiven pulses as same time, thus, the MLP has large error
and learns diﬀerent points.

Table 1: Learning performance.
Avg. Err.
Min.
Max.
St. Dev.
0.00290.0001 0.0605 0.0079
0.00080.0001 0.0202 0.0022
0.00170.0001 0.0209 0.0037
0.00120.0001 0.0603 0.0048

(1)
(2)
(3)
(4)

Next, we show the learning curve of the MLPs. We cansee that the MLP with pulse glial chain (D = 1) has thebest learning performance, and that the conventional MLP
and the MLP with chain glial chain (D = 0) converged
early and can not reduce the error. We consider that theconventional MLP and the MLP with pulse glial chain (D =1) are trapped local minimum. Comparing results, the MLP
with pulse glial chain (D = 1) can escape out from the localminimum.

Figure 4: Skew tent map.
The BP learning for the MLPs is carried out by giving
four successive points of the chaotic time series as an input and the following 0 or 1 as an output. The learning
is repeated for 200 diﬀerent sets which are included two
diﬀerent chaotic time series like Fig. 5.

1.0000Conventional MLP
MLP with chain pulse glial network (D=1)

0.1000

MLP with chain pulse glial network (D=0)
MLP with random timing pulseMS
E 0.0100(a) Step 1.

(b) Step 2.

(c) Step 3.

0.0010Figure 5: Learning procedure.

3.2. Simulation results

0.0001First, we show statistic results which are an average oferror (Avg.), a minimum error as 200 trials (Min.), a maximum error as 200 trials and a standard deviation of errors
in Table 1. The MLPs learn chaotic time series of diﬀerent initial conditions. As one trial, the MLPs learn during- 437 -010000

20000

30000

40000

Learning time

Figure 6: Learning curves by the MLPs.

50000

3.3. Parameters dependency
Finally, we show that the parameters dependency of theMLP with pulse glial chain. We change the number of delay D and the weight of glia output α. Figure 7 is the average of error by 100 trials as each condition. From thisfigure, the error is small when D are 1 and 9. We consider that the MLP need above a certain energy for escaping out from the local minimum. In D = 1, the MLP isgiven large energy, because, the glial chain generates pulse
as very similar timing. And also, in D = 9, the MLP isgiven large energy by three glias, because, the pulse is delayed as propagating into the glial chain, and the first glia
generates next pulse, which is shown in Fig. 8.

first glia’s pulse output excites neighborhood glia, after thatthe neighborhood glia generates similar patter pulse andthis process is continued. Thus, the pulse looks like propagating into the glial chain.
The MLP with pulse glial chain is better than the conventional MLP for learning the chaotic time series. Because
the MLP with pulse glial chain has the pulse by the glia, itis helped that the MLP escaped from the local minimum.
After that we researched the parameter dependency of ourMLP. In this result, we found two characteristic parameterswhich are D = 1 and D = 9. We consider that the MLP
can be obtained above a certain energy by the glial chain.
Because the pulses are generated at similar timing as theseconditions.
Acknowledgment−3x 10This work was partly supported by The Nakajima Foundation.

876MSE

5References

43[1] P.G. Haydon, “Glia: Listening and Talking tothe Synapse,” Nature Reviews Neuroscience, vol. 2,pp. 844-847, 2001.

210108160.8
0.6

4

D0.4

20.20

α0Figure 7: Parameter dependency of the pulse glial chain.

[2] S. Koizumi, M. Tsuda, Y. Shigemoto-Nogami andK. Inoue, “Dynamic Inhibition of Excitatory Synaptic
Transmission by Astrocyte-Derived ATP in Hippocampal Cultures,” Proc. National Academy of Science ofU.S.A, vol. 100, pp. 11023-11028,[3] C. Ikuta, Y. Uwate and Y. Nishio, “Chaos Glial Network Connected to Multi-Layer Perceptron for Solving
Two-Spiral Problem,” Proc. ISCAS’10, pp. 1360-1363,
May 2010. 2003.

0.5
0.4

[4] D.E. Rumelhart, G.E. Hinton and R.J. Williams,“Learning Representations by Back-Propagating Errors,” Nature, vol. 323-9, pp. 533-536, 1986.

0.3
0.2
0.1
0

[5] S. Ozawa, “Role of Glutamate Transporters in Excitatory Synapses in Cerebellar Purkinje Cells,” Brain andNerve, vol. 59, pp. 669-676, 2007.

975311

1121

31

41

51

61

71

8191

141
121 131101 111

Figure 8: Propagating pulse by the pulse glial chain as (D =9).

4. Conclusion
In this study, we have proposed the MLP with pulse glialchain. This MLP is inspired from the biological nervoussystems. In the pulse glial chain, the first glia generates
pulse output exciting by the connecting neuron output. The[6] G. Perea and A. Araque, “Glial Calcium Signaling andNeuro-Glia Communication,” Cell Calcium, vol. 38,375-382, 2005.
[7] S. Kriegler and S.Y. Chiu, “Calcium Signaling of Glial
Cells along Mammalian Axons,” The Journal of Neuroscience, vol. 13, 4229-4245, 1993.
[8] M.P. Mattoson and S.L. Chan, “Neuronal and Glial
Calcium Signaling in Alzheimer’s Disease,” Cell Calcium, vol. 34, 385-397, 2003.

- 438 -