Investigation of Multi-Layer Perceptron with
Propagation of Glial Pulse to Two Directions
Chihiro Ikuta

Yoko Uwate

Yoshifumi Nishio

Dept. of Electrical and Electronics Eng., Dept. of Electrical and Electronics Eng., Dept. of Electrical and Electronics Eng.,
Tokushima University
Tokushima University
Tokushima University
2–1 Minami–Josanjima,
2–1 Minami–Josanjima,
2–1 Minami–Josanjima,
Tokushima Japan
Tokushima Japan
Tokushima Japan
Email: ikuta@ee.tokushima-u.ac.jp
Email: uwate@ee.tokushima-u.ac.jp
Email: nishio@ee.tokushima-u.ac.jp

Abstract— A glia is nervous cell which exists in a brain. The
glia can transmit signal to other glias and neurons by change ofions’ densities. We have an interest in this feature of the glia.
We consider that we can apply this feature to an artificial neuralnetwork.
In this study, we propose a Multi-Layer Perceptron (MLP)
with propagation of glial pulse to two directions. The proposed
MLP has the glias in a hidden layer. The glias are connected withneurons and are excited by the outputs of neurons. The exciting
glias generate pulses and the pulses affect neurons’ thresholds
and neighboring glias. We consider that the MLP obtains therelationships of position of neurons in the hidden layer and thisinformation give good influence to the MLP leaning. We confirm
that the proposed MLP has better learning performance than theconventional MLP. Moreover, we confirm that the performance ofthe proposed MLP is changed by some conditions of propagationof the glial pulse.

I. I NTRODUCTION
Applications of neurons have been investigated for a longtime. However, the glias have not attracted attentions, because
the glias can not use an electric signal. Recently, someresearchers discovered that the glias transmitted signals byusing the ions [1][2]. The glias are considered to be important
for the brain works. Especially, Ca2+ can change a membranepotential of the neuron. We consider that the glias can beapplied to the artificial neural network.
In the previous study, we proposed a pulse glial chain fromthe features of the biological glia [3]. In that model, one
glia generates a pulse to be excited by the output of theconnecting neuron. After that, the next glia is excited andgenerates the pulse. The pulse propagates to other glias by
the glial chain. We connected the pulse glial chain to theMulti-Layer Perceptron (MLP). We confirmed that the MLP
with pulse glial chain had better learning performance thanthe conventional MLP. However, we assumed that the only
one glia was excited by the output of the connecting neuron,and that the pulse propagated to one direction.
In this study, we propose MLP with propagation of glial
pulse to two directions. All glias are connected with theneurons and are excited by outputs of the connecting neurons.
The generated pulses excite glias on both sides. We connectthe glias to the neurons in a hidden layer and the outputsof the glias influence the threshold of neurons. Because the978-1-4673-0219-7/12/$31.00 ©2012 IEEE2099

Ca2+ can change the membrane potential of the neuron in
biological system [5][6]. We consider that the glias give therelationships of position of neurons in the hidden layer. By
the simulations, we show the learning performance and theparameters dependency of the proposed MLP.
II. P ROPOSED M ETHOD
The MLP is the most famous feed forward neural network.
This network is composed of layers of neurons and its outputs
are controlled by the weights of connections. In general itis learned by Back Propagation algorithm (BP) which wasproposed by D.E. Rumelhart [4]. We connect the glias to theneurons in the hidden layer. We show the proposed MLP inFig. 1.

Fig. 1.

MLP with propagation of glial pulse to two directions.

A. Glial Pulse
Currently, the glia attracts researchers attentions in the
biological or the medical fields. Because the glia has several
important works in the brain. Especially, the glia can changethe Ca2+ density and this Ca2+ density become pulse response[7][8]. The Ca2+ change the threshold of the neuron. Moreover, this influence propagates to a wide range in the brain.
In this study, we propose the glial pulse which propagatesto two directions. The glial pulse is inspired from the features

of biological glia. All glias are connected the neurons in
the hidden layer and generate pulses by the outputs of theconnecting neurons. We define an output function of the gliain Eq. (1).
𝜓𝑖 (𝑡 + 1) ={1, {(𝜃𝑛 < 𝑦𝑖 ∪ 𝜓𝑖+1,𝑖−1 (𝑡 − 𝑖 ∗ 𝐷) = 1)
∩ (𝜃𝑔 > 𝜓𝑖 (𝑡))} ,𝛾𝜓𝑖 (𝑡), 𝑒𝑙𝑠𝑒,(1)

where 𝜓 is an output of a glia, 𝛾 is an attenuated parameter,
𝑦 is an output of a connecting neuron, 𝜃𝑛 is a glia thresholdof excitation, 𝜃𝑔 is a period of inactivity, and 𝐷 is a delay
time of a glial effect. The glias do not learn by BP algorithm,
however, the neurons are learned by BP algorithm. Thereby,
the generation pattern of glial pulse dynamically changes
during the MLP learning.
Figure 2 is an example of glial pulses for obtained from thesimulations. During (a) in the figure, the 2nd glia is excited,
after that the 1st glia and the 3rd glia are excited by the effectof the 2nd glia. The 2nd glial effect propagate to other glias,
however, the 9th glia excite own before to be propagated theeffect of the 2nd glia. Thereby, the pattern of pulse is notclearly stepwise. In the center of (a), the 3rd glia is excited.
This effect propagate to the 2nd glia, however, this effect does
not propagate to the 4th glia. Because the 4th glia has theperiod of inactivity when the 3rd glial effect propagate to the4th glia. During (b), the 6th glia is excited first. The 6th glialeffect propagates to its neighboring glias. We can see that thegeneration pattern of glial pulses dynamically changes during
MLP learning.
B. Updating Rule of Neuron
The neuron has multi-inputs and single output. We canchange the neuron output by tuning weights of connections.
The standard updating rule of the neuron is defined by Eq. (2).
⎛⎞𝑛∑𝑤𝑖𝑗 (𝑡)𝑥𝑗 (𝑡) − 𝜃𝑖 (𝑡)⎠ ,(2)𝑦𝑖 (𝑡 + 1) = 𝑓 ⎝𝑗=1

where 𝑦 is an output of the neuron, 𝑤 is a weight ofconnection, 𝑥 is an input of the neuron, and 𝜃 is a threshold ofneuron. In this equation, the weights of the connections andthe thresholds of neurons are learned by BP algorithm. Next,
I show a proposed updating rule of the neuron. We add theglial effect to the threshold of neuron. This updating rule isused to the neurons in the hidden layer and is described byEq. (3).
⎛⎞𝑛∑𝑤𝑖𝑗 (𝑡)𝑥𝑗 (𝑡) − 𝜃𝑖 (𝑡) + 𝛼𝜓𝑖 (𝑡)⎠ ,(3)𝑦𝑖 (𝑡 + 1) = 𝑓 ⎝𝑗=1

where 𝛼 is weight of glial effect. We can control the glial effectby changing 𝛼. In this equation, the weight of connection
and the threshold are learned by BP algorithm as same as
the standard updating rule of the neuron. However, we fix theglial effect 𝛼 in this algorithm.

2100

Fig. 2.

An example of glial pulses (𝐷 = 5).

Equations (2) and (3) use a sigmoidal function as theactivating function which is described by Eq. (4).
1𝑓 (𝑎) =.
(4)
1 + 𝑒−𝑎III. S IMULATIONIn this section, we confirm the learning performance of theproposed MLP. Moreover, we show features of the proposed
MLP for some conditions. We use five different MLPs, whichare(1)
The conventional MLP(2)
The MLP with random timing pulses(3)
The MLP with same timing pulses(4)
The MLP with pulse glial chain(5)
The MLP with propagation of glial pulse to twodirections (proposed)
In the MLP with random timing pulses, this MLP is given
the pulses to the thresholds of neurons at random timing.
The random timing pulses are decorrelated each other. TheMLP with same timing pulses is given the pulses which aregenerated at same timing.
A. Simulation Task
The MLPs learn the classifications of two different chaotictime series. Both chaotic time series are generated by a skew
tent map. The skew tent map is defined by Eq. (5).
{ 2𝜙(𝑡)+1−𝐴 (−1 ≤ 𝜙(𝑡) ≤ 𝐴)
𝜙𝑖 (𝑡 + 1) =1+𝐴

−2𝜙(𝑡)+1+𝐴
1−𝐴,(𝐴 < 𝜙(𝑡) ≤ 1)

(5)

The oscillation pattern of the skew tent map is changed by𝐴. We normalize the chaotic time series of the skew tent map
between 0 and 1. Because our MLP use the value between 0and 1. Figure 3 is one of the learning data.

Fig. 3.

TABLE IL EARNING PERFORMANCE .

(1)
(2)
(3)
(4)
(5)

Avg.
0.00287
0.00271
0.00279
0.00252
0.00238Min.
0.000020.00001
0.000020.00001
0.00001Max.
0.10026
0.075200.07537
0.07518
0.02517

St. Dev.
0.008020.00566
0.00579
0.00500
0.00443

One of the learning data.

B. Simulation Results
In this simulation, the MLPs learn 50000 times during theone trials. We use 10 pairs of different 𝐴. We simulate the 100
trials from different initial weights of connections for each pairof different 𝐴. We use the Mean Square Error (MSE) to as ameasure of performance. It is described by Eq. (6).
𝑁1 ∑(𝑇𝑛 − 𝑂𝑛 )2 ,𝑀 𝑆𝐸 =𝑁 𝑛=1(6)

where 𝑁 is a number of learning data, 𝑇 is a target value, and𝑂 is an output of MLP.
1) Learning Performance: We compare the five different
MLPs. We show the results of MLPs in Table I. From thistable, we can see that the learning performance of the proposed
MLP is the best of all. The learning performance of theconventional MLP is the worst. The leaning performance ofthe MLP with same timing pulses is worse than the MLP
with random timing pulse. We consider that the MLP withsame timing pulses is given too much energy from the glias.
The proposed MLP and the MLP with pulse glial chain havebetter learning performance than the MLP with random timingpulses. We consider that the proposed MLP and the MLP
with pulse glial chain have the information of the position ofneurons. In the proposed MLP, all glias can generate pulses by
outputs of each connecting neuron. Thus, the proposed MLP
can change pattern of pulses more dynamically than the MLP
with pulse glial chain.
Figure 4 is an example of learning curves of the MLPs.
These learning curves are similar result to the Table I. Moreover, the convergence of the proposed MLP is the fastest ofall.
2) Parameters Dependency: Next, we show a change ofthe learning performance of the proposed MLP by someparameters. Figure 5 is the change of the learning performancefor different 𝛼 and 𝜃𝑔 in Eq. (1). 𝛼 is the weight of the glial
effect and 𝜃𝑔 is the period of inactivity. When 𝜃𝑔 becomes

2101

Fig. 4.

An example of leaning curves of MLPs.

lager, the generation span of the pulse of the glia becomes
longer. From this result, the MLP has the high acceptableamount of 𝛼 as lager 𝜃𝑔 . Because the MLP can learn the true
target value during the period of inactivity. In the large 𝜃𝑔 , theperiod of inactivity is long, thus, the MLP receives the large
glial effect, it can learn the true target value.
Next, we show a learning performance for each 𝜃𝑛 in Fig. 6.
𝜃𝑛 is glial threshold of excitation. When the 𝜃𝑛 is small, theglia can be excited easily. However, the glias can not change
the pattern of pulses, because most of glias are excited by
the connecting neurons. From this figure, we can see that theMLP has high learning performance as 𝜃𝑛 between 0.8 and0.9. When 𝜃𝑛 is 0.95, the the MSE becomes large. The MLP
cannot obtain enough energy from glias.
Figure 7 is a learning performance of the proposed MLP fordifferent 𝐷. 𝐷 is a delay time of glial effect. If the 𝐷 becomeslarger, the propagated pulses are generated more different
timing. From this result, the MLP learning performance ishigh as 𝐷 = 1 and 𝐷 = 9. In the case of 𝐷 = 1, the pulses
are generated delay 1 leaning time. Near glias generate pulses
at similar timing and the MLP is given high energy at one
learning time. In the case of 𝐷 = 9, two glias are sometimes
excited at the same timing. Because the glias finish the periodof inactivity earlier than propagating pulses from far glias. Theglias which finished the period of inactivity can generate thenew pulses.

Fig. 5.

Fig. 6.

Fig. 7.

Leaning performance as different 𝐷.

Fig. 8.

Leaning performance as different 𝛾.

Leaning performance between 𝛼 and 𝜃𝑔 .

Leaning performance as different 𝜃𝑛 .

Finally, we show the learning performance of the MLP fordifferent 𝛾. 𝛾 is the parameter of attenuated pulse. When the𝛾 becomes smaller, the pulses are attenuated more rapidly.
For small 𝛾, glial effects appeared when neighboring glias
generated pulses. We consider that the learning performancesare worse for small 𝛾. Moreover, the leaning performance
becomes high when 𝛾 is large value. In the case of 𝛾 = 9,
the MLP has the highest learning performance. However,the learning performance becomes worse for 𝛾 = 0.95. We
consider that the attenuation of the pulses are too slowly,
thereby, the outputs of the glias cannot become pulses.

IV. C ONCLUSIONSIn this study, we have proposed the MLP with propagationof glial pulse to two directions. It is inspired from features ofbiological glia. All glias generate the pulse when the gliasare excited by the outputs of the connecting neurons. Thepulses are propagated by glia, after that, the pulses influence
the threshold of neurons. We connected the glias to the neurons
in the hidden layer of MLP. We confirmed that the MLP
with propagation of glial pulse to two directions had better

2102

learning performance than the conventional MLP. From thesimulations, we showed that the MLP obtained the high learning performance by to be given relationships of position ofneurons. Moreover, we investigated dependency of parametersof proposed MLP. We confirmed that the learning performanceof proposed MLP could be controlled by changing parameters.
R EFERENCES[1] P.G. Haydon, “Glia: Listening and Talking to the Synapse,” NatureReviews Neuroscience, vol. 2, pp. 844-847, 2001.
[2] S. Koizumi, M. Tsuda, Y. Shigemoto-Nogami and K. Inoue, “DynamicInhibition of Excitatory Synaptic Transmission by Astrocyte-Derived
ATP in Hippocampal Cultures,” Proc. National Academy of Science ofU.S.A, vol. 100, pp. 11023-11028, 2003.
[3] C. Ikuta, Y. Uwate and Y. Nishio, “Multi-Layer Perceptron with Pulse
Glial Chain,” Proc. NOLTA’11, pp. 435-438, Sep. 2011.
[4] D.E. Rumelhart, G.E. Hinton and R.J. Williams, “Learning Representations by Back-Propagating Errors,” Nature, vol. 323-9, pp. 533-536,
1986.
[5] S. Ozawa, “Role of Glutamate Transporters in Excitatory Synapses inCerebellar Purkinje Cells,” Brain and Nerve, vol. 59, pp. 669-676, 2007.
[6] G. Perea and A. Araque, “Glial Calcium Signaling and Neuro-Glia
Communication,” Cell Calcium, vol. 38, pp. 375-382, 2005.
[7] S. Kriegler and S.Y. Chiu, “Calcium Signaling of Glial Cells along
Mammalian Axons,” The Journal of Neuroscience, vol. 13, pp. 42294245, 1993.
[8] M.P. Mattoson and S.L. Chan, “Neuronal and Glial Calcium Signalingin Alzheimer’s Disease,” Cell Calcium, vol. 34, pp. 385-397, 2003.