Improvement of Learning Performance ofMulti-Layer Perceptron by Two Different
Pulse Glial Networks
Chihiro Ikuta†, Yoko Uwate†, Yoshifumi Nishio†, and Guoan Yang‡†Dept. of Electrical and Electronics Eng., Tokushima University
2–1 Minami–Josanjima, Tokushima Japan
Email: {ikuta, uwate, nishio}@ee.tokushima-u.ac.jp‡Dept. of Automation Science and Technology,
School of Electronic and Information Eng., Xi’an Jiaotong University
No.28 Xianning West Road, Xi’an City, Shaanxi Province, China
Email: gayang@mail.xjtu.edu.cn

Abstract— A glia is the most number of nervous cells in abrain. The glia is investigated in a medical field, because the glia
correlates to neuron works and composes a human cerebration.
We consider that the glia function can be applied to an artificial
neural network. In this study, we propose the Multi-Layer
Perceptron (MLP) with the two different pulse glial networks.
The proposed MLP has the glial network which is inspired from
biological functions of the glia. One neuron is connected with
two glias. Two glias generate the pulse depending on the output
neurons. One glia connects the neuron for increasing the outputof neuron. On the other hand, the glia connects the neuron fordecreasing the output of neuron. Both glias composes the glialnetworks. These effects are propagated into the networks. The
glial effects become complexity and affects the MLP learning
performance. By the computer simulation, we confirm that thelearning performance of the proposed MLP is better than theconventional MLP.

In this study, we propose the Multi-Layer Perceptron (MLP)
with two different pulse glial networks. One neuron connectswith two glias. Two glias have different excitation thresholds
and inverse value output. Glia1 gives the pulse to the neuron
for increasing the neuron output. Glia2 gives the pulse to theneuron for decreasing the neuron output. When the neuron haslarge the output, the glia is excited and generate the pulse.
First, glia1 is excited because the glia1 ’s excitation thresholdis smaller than the glia2 . By the glial effect, the neuron output
becomes larger more. The glia2 is excited when the neuron
output is over the excitation threshold of the glia2 . We consider
that the glial effect give the energy to the MLP for escaping
out from the local minimum. By the computer simulation,
we confirm that the proposed MLP obtain the better learning
performance than the conventional MLP.

I. I NTRODUCTION

II. P ROPOSED MLP

Applications of neurons have been investigated for a longtime. However, the glias have not attracted attentions, because
the glias can not use an electric signal. Recently, someresearchers discovered that the glias transmitted signals byusing the ions such as Ca2+ , Glu, ATP, and so on [1][2].
The glias are considered to be important for the brain works.
Especially, Ca2+ can change a membrane potential of theneuron [3][4]. Moreover, the Ca2+ affects the states of theneighboring glias. The Ca2+ effects are propagated to wide
range in the brain [5][6]. We consider that the glias can beapplied to the artificial neural network.
In the previous study, we proposed the positive and negative
pulse glial chain [7]. In this network, the glias are connectedwith the neurons one by one. When the neuron has large
output, the connected glia generates the positive value pulse.
When the neuron has small output, the connected glia generates the negative value pulse. The generated pulses influence
the neuron threshold and neighboring glial state The positive
and negative pulse glial chain influences the neuron thresholdfor increasing the neuron output.

The MLP is a famous feed forward neural network. It is
composed of the layer of neurons. It can solve various problem
such as a classification problem, a pattern recognition, a data
mining, and so on. We can change the output of the networkby tuning the weight of connections. We often use a BackPropagation (BP) algorithm for the deciding method of theweight of connections [8]. However, the BP algorithm often
falls into a local minimum, because it use the steepest decent
method. For the local minimum problem, we exploit the noise.
The noise gives the energy for escaping out from the local
minimum.
In this study, we propose the MLP with two different
pulse glial networks. The construct of the proposed MLP isillustrated in Fig. 1. The two different pulse glial networks
are composed of the glias. In this network, we connect twoglias to one neuron. The glias generate the pulse dependingon the neuron output. The glial pulse is inputed to theneuron threshold. We consider that the glial pulse give theenergy to the MLP for escaping out from the local minimum.
Moreover, the pulses are propagated to the neighboring glias.

978-1-4577-1729-1/12/$26.00 ©2012 IEEE.

356

We consider that the amount of pulse is changed little bylittle in the neurons. We believe that this work gives the novel
relationships in the hidden layer of the MLP.

Moreover, the glial network should change the work dependingon the MLP learning.
In two different pulse glial networks, we connect two glias
to one neuron. Two glias make different networks. One glia
can easily excite by the neuron output. On the other hand, theglia need the large neuron output for excitation.
𝜓𝑖 (𝑡 + 1) ={1, {(𝜃𝑛𝑖 < 𝑦𝑖 ∪ 𝜓𝑖+1,𝑖−1 (𝑡 − 𝑖 ∗ 𝐷) = 1)
∩ (𝜃𝑔 > 𝜓𝑖 (𝑡))} , (3)𝛾𝜓𝑖 (𝑡), 𝑒𝑙𝑠𝑒,

Neuron…

𝜓𝑖 (𝑡 + 1) ={−1, {(1 − 𝜃𝑛𝑖 < 𝑦𝑖 ∪𝜓𝑖+1,𝑖−1 (𝑡 − 𝑖 ∗ 𝐷) = 1) ∩ (𝜃𝑔 > 𝜓𝑖 (𝑡))} , (4)𝛾𝜓𝑖 (𝑡), 𝑒𝑙𝑠𝑒,GliaFig. 1.

The MLP with two different pulse glial networks.

A. Updating rule of neuron
The neuron has multi-inputs and single output. We canchange the neuron output by the tuning weights of connections. The standard updating rule of the neuron is defined byEq. (1).
⎛⎞𝑛∑𝑤𝑖𝑗 (𝑡)𝑥𝑗 (𝑡) − 𝜃𝑖 (𝑡)⎠ ,(1)𝑦𝑖 (𝑡 + 1) = 𝑓 ⎝𝑗=1

where 𝑦 is an output of the neuron, 𝑤 is a weight ofconnection, 𝑥 is an input of the neuron, and 𝜃 is a thresholdof neuron. We connect the two different pulse glial networks
to the hidden-layer of the MLP. Two glial pulses input to theneuron threshold, because the biological glial effect affects tothe membrane potential of neuron. We define the proposed
neuron updating rule in Eq. (2).
𝑦𝑖 (𝑡 + 1) =⎛⎞𝑛∑𝑓⎝𝑤𝑖𝑗 (𝑡)𝑥𝑗 (𝑡) − 𝜃𝑖 (𝑡) + 𝛼(𝜓𝑖1 (𝑡) + 𝛽𝜓𝑖2 (𝑡))⎠(2),
𝑗=1

where Ψ1 and Ψ1 are the outputs of the glias, 𝛼 is a weightof the glial effect, 𝛽 is a weight of 𝑃 𝑠𝑖2 . By 𝛽, the amountratio 𝜓𝑖 1 and 𝜓𝑖 2 are changed.
B. Two different pulse glial networks
The glia is one of nervous cells in the brain. It transmits
signals by ion concentrations. We note this function and wehave proposed the glial network. In this study, we extend theglial network to two different pulse glial networks. The glial
network is connected with the neurons in the hidden layerof the MLP. It gives the energy for increasing the neuron
output. In this model, we sometimes observed that the MLP
learning is early fixed. We hope that the glial network help
the MLP, however the glial network gives constant amountof pulse. Thus, the MLP cannot escape out from the local
minimum. However, the MLP cannot learn when the glial
effect is strong. From this result, we consider that the MLP
needs more energy for escaping out from the local minimum.

where 𝜓 is an output of a glia, 𝛾 is an attenuated parameter, 𝑦is an output of a connecting neuron, 𝜃𝑛 is a glia threshold ofexcitation, 𝜃𝑔 is a period of inactivity, and 𝐷 is a delay time ofa glial effect. The glia does not learn, it depend on the output
of connected neurons. However, the neurons are learned byBP algorithm, thus the generation pattern of glia output candynamically change during the learning. In the two different
pulse glial networks, two glial networks have different 𝜃𝑛 . We
define 𝜃𝑛1 < 𝜃𝑛2 in every simulation. Thus, the connected twoglias are excited different conditions. We believe that the pulse
glial network which has 𝜃𝑛2 help the proposed MLP when theproposed MLP is trapped into local minimum. We present an
example of the pulses by two glias in Fig. 2. In this case,the glia 1 only generates the negative pulses. However, theglia 2 generates the positive and the negative pulses. Thus,
the neuron receives the positive and the negative pulses. In
the previous glial network, the pulse often becomes like glia
1. We consider that the pulse pattern become complexity byconnected two glial network.
Figure 3 is an example of the inputed glial effect which isadded two glial pulses. We show the minimum output to theblack, the large output is drawn in the white, and the gray isnear 0. The generated glial effect patterns are dynamicallychange, because two glias have the inversed work for theneuron output. Thus, neuron output is changed by the glial
effects. Moreover, the neurons are learned by the BP algorithm
and the neurons’ outputs are changed. We can see that thepulses are propagated into the glial network. The pulses are
sometime disappeared, because two connected glias annihilate
each other. In the case of dark black or light white, two glias
generate same vector values. Thus, they are turned up eacheffect.
III. SIMULATIONIn this simulation, we use Two-Spirals Problem (TSP) to thebenchmark shown in Fig. 4. The TSP is a famous benchmark
for the artificial neural network. Moreover, the TSP is known
the high non-linearity problem, thereby the standard MLP
cannot easily solve this problem. The MLP learns two different
spirals coordinates and classifications of spirals. Moreover,357

the learned MLP can classier non-learned coordinates to twospirals. The ideal classification result is presented in Fig. 4 (b).
We compare the MLP with two different pulse glial networks

ψ1(t)210-1-202004006008001000

02004006008001000

ψ2(t)

210-1

ψ1(t)+ψ2(t)-2(a) The learning data.

(b) The ideal result

21Fig. 4.

Two-spirals problem.

0-1-20200

400

600

800

1000

Iterations (t)Fig. 2.

to the MLP with pulse glial chain (two directions). The MLPs
are composed 2-40-1 neurons. The MLPs learn 50000 times
in 1 trial. We obtain the average of error from 100 trials. Forthe measure, we use the Mean Square Error(MSE) which is
described by Eq. (5).

An example of the glial pulse.

𝑁Iteration (t)𝑀 𝑆𝐸 =i with numbers of 1 and 40 on the axis
Fig. 3.

The inputted glial effects into 40 neurons.

1 ∑(𝑇𝑛 − 𝑂𝑛 )2 ,𝑁 𝑛=1(5)

where T is a target value, O is an output of the MLP.
Figure 5 shows the approximation results for different parameters. In the this figure, the horizontal axis is weight of glial
effect 𝛼 and the vertical axis is the MSE. We decide 𝜃𝑛1 = 0.8
which is not changed and we change 𝜃𝑛2 = 0.9, 0.95𝑎𝑛𝑑0.99.
In the case of 𝛽 = 0.5, the inverse effect is small, thus thecurves are similar to the previous MLP. And also, the three
curves of the proposed MLP are similar. Because, the influenceof the inverse glial network is smaller then another glialnetwork. However, when 𝛽 becomes high, the approximation
performance of proposed MLP is better than the previousMLP. The proposed MLP has a better performance than theprevious MLP at 𝛽 = 1.0. Then, the curves of proposed
MLP are similar trends to the previous MLP, however theerrors are reduced in every 𝛼. In Fig 5 (c), the approximation
performance decreased when 𝛽 is higher than 0.8 Because theglial effect become too large by 𝛽. We consider that the MLP
cannot learn the ideal point by the glial effects.
Next, we present the classification performance in eachcondition. In this result, we input the coordinates in 𝑥 − 𝑦field to after learning MLP. After that the MLP obtain theoutput. We compare the ideal classification (Fig. 4 (b)) tothe MLP output. The classification performances have similar
trend to the approximation performances. In 𝛽 = 0.5, theclassification performances of the proposed MLP are similar tothe previous MLP. In 𝛽 = 1.0, the classification performancesof the proposed MLP are better than the previous MLP. The
proposed MLP is worth classification performance than theprevious MLP when the glia has 𝛽 = 2.0 and 𝛼 is high.
From these results the two pulse glial network give the good
influence to both abilities of the MLP.

358

IV. C ONCLUSIONS(a) 𝛽 = 0.5.

(b) 𝛽 = 1.0.

We have proposed MLP with two different pulse glialnetworks. The proposed network is inspired from the functionsof the biological glia. In the proposed network, we connected
two glias to one neuron. And also, two glias make the different
glial networks which have different parameters. The glia isfired by the connecting neuron output, after that the fired glia
generates the pulse. The pulse affects the neuron threshold
and the state of neighboring glias. Thus, the glial pulses
are propagated into the glial network. We considered thatthe pulses are affected each other, thereby the pulses pattern
become more complexity than the previous network. We
confirmed that the proposed MLP has the better performances
than the previous MLP in some conditions.
ACKNOWLEDGMENT
This work was partly supported by MEXT/JSPS Grant-inAid for JSPS Fellows (24⋅10018).
R EFERENCES[1] P.G. Haydon, “Glia: Listening and Talking to the Synapse,” NatureReviews Neuroscience, vol. 2, pp. 844-847, 2001.
[2] S. Koizumi, M. Tsuda, Y. Shigemoto-Nogami and K. Inoue, “DynamicInhibition of Excitatory Synaptic Transmission by Astrocyte-Derived
ATP in Hippocampal Cultures,” Proc. National Academy of Science ofU.S.A, vol. 100, pp. 11023-11028, 2003.
[3] S. Ozawa, “Role of Glutamate Transporters in Excitatory Synapses inCerebellar Purkinje Cells,” Brain and Nerve, vol. 59, pp. 669-676, 2007.
[4] G. Perea and A. Araque, “Glial Calcium Signaling and Neuro-Glia
Communication,” Cell Calcium, vol. 38, pp. 375-382, 2005.
[5] S. Kriegler and S.Y. Chiu, “Calcium Signaling of Glial Cells along
Mammalian Axons,” The Journal of Neuroscience, vol. 13, pp. 42294245, 1993.
[6] M.P. Mattoson and S.L. Chan, “Neuronal and Glial Calcium Signalingin Alzheimer’s Disease,” Cell Calcium, vol. 34, pp. 385-397, 2003.
[7] C. Ikuta, Y. Uwate and Y. Nishio, “Multi-Layer Perceptron with Positive
and Negative Pulse Glial Chain for Solving Two-Spirals Problem,” Proc.
IJCNN’12, pp. 2590-2595, Jun. 2012.
[8] D.E. Rumelhart, G.E. Hinton and R.J. Williams, “Learning Representations by Back-Propagating Errors,” Nature, vol. 323-9, pp. 533-536,
1986.

(c) 𝛽 = 2.0.

Fig. 5.

Approximation results.

(a) 𝛽 = 0.5.

(b) 𝛽 = 1.0.

(c) 𝛽 = 2.0.

Fig. 6.

Classification results.

359