2014 International Joint Conference on Neural Networks (IJCNN)
July 6-11, 2014, Beijing, China

Investigation of Multi-Layer Perceptron with Pulse Glial Chain
Based on Individual Inactivity Period
Chihiro Ikuta, Yoko Uwate, and Yoshifumi Nishio

Abstract— In this study, we propose a Multi-Layer Perceptron (MLP) with pulse glial chain based on individual inactivity
period which is inspired from biological characteristics of a glia.
In this method, we one-by-one connect a glia with neurons in
the hidden-layer. The connected glia is excited by the connecting
neuron output. Then, the glia generates the pulse. This pulse is
input to the connecting neuron threshold. Moreover, this pulse ispropagated into the glia network. Thus, the glia has a position
density each other. In this network, a period of inactivity of
the glia is dynamically changed according to pulse generation
time. In the previous method, we fix the period of inactivity,
thus the pulse generation pattern is often fixed. It is similar tothe local minimum. By varied the period of inactivity, the pulse
generation pattern obtains the diversity. We consider that thisdiversity of the pulse generation pattern is efficiency to the MLP
performance. By the simulation, we confirm that the proposed
MLP improves the MLP performance than the conventional
MLP.

I. I NTRODUCTION
Human brain has two kinds of nervous cells which are
the neuron and the glia. We have considered that ahuman cerebration is only made by the neurons. Because
the neuron can transmit an electric signal each other andthis phenomenon was found at an earlier stage of a research.
Actually, the transmission of the electric signal has a highrelationship for the human cerebration and it achieved somepositive results. On the other hand, we considered that theglia was a support cell for the neuron. However, someresearchers discovered that the glia has novel glia functions[1][2]. The glia can transmit signal by using ions concentrations which are a glutamate acid, an adenosine triphosphoricacid (ATP), calcium (Ca2+ ), and so on [3][4]. These ionsare also used in a gap junction of the neuron. Among them,
the Ca2+ is important for the transmission of information
between the glia. The concentration change of the Ca2+
induces the stimulus from the neuron. The Ca2+ propagatesto the other glias. The glia is considered that the glia and theneuron closely related. Moreover, the glia makes the different
network from the neuron. Currently, we should consider toa network between the neuron and the glia.
The glia-neural network is important for a detailed investigation of the brain works. However, the brain research ismainly about the neuron. Especially, the application of theglia has not almost investigated. We therefore applied theglia characteristics to a Multi-Layer Perceptron (MLP) for theA

Chihiro Ikuta, Yoko Uwate, and Yoshifumi Nishio are with Department ofElectrical and Electronics Engineering, Tokushima University, Japan (email:{ikuta, uwate, nishio}@ee.tokushima-u.ac.jp).
This work was partly supported by MEXT/JSPS Grant-in-Aid for JSPS
Fellows (24⋅10018).

978-1-4799-1484-5/14/$31.00 ©2014 IEEE

1638

application of the glia. The MLP is a famous artificial neuralnetwork. This network is composed of layers of neurons.
The MLP is generally learned by a Back Propagation (BP)algorithm [5]. By this learning, the MLP can be applied to apattern learning, a data mining, and so on. However, the BP
algorithm has a local minimum problem because this learningalgorithm uses the steepest decent method. The MLP does
not have the connections in the same layer. The neurons
connect to different layer of neurons thus the neurons do
not correlate in the same layer. In the previous study, we
proposed the MLP with pulse glial chain in IJCNN’12 [6].
In the previous model, we connect the glia with the neurons
for solving these problems. The glia is connected with theneurons in the hidden-layer and the neighboring glias, andit generates the pulse according to the connecting neuron
output. The generated pulse is propagated to the connecting
neuron and the other glias. We consider that the glia pulse
gives position relationships of the neuron in the hidden-layer
and an energy for escaping out from the local minimum.
From the previous study, we confirmed that the previous
model has a better performance than the standard MLP.
However, the previous model has a problem. This problem
is that the pulse generation pattern is often converged in theprevious model. Every glia has the same parameters. Thereby
whole pulse generation pattern is depended on the one glia
influence.
In this study, we propose the MLP with pulse glial chain
based on individual inactivity period. We introduce the individual period of inactivity to each glia. If the glia is excitedby the connecting neuron output, the glia cannot be excited
again during the period of inactivity. The previous model
has same time length of the period of inactivity, thereby
the generation pulse pattern becomes the same cycle. In thismethod, the time length of the period of inactivity is varied toa short when the glia is continuously excited. The glia whichis excited at short interval, obtains different pulse generation
cycle. We consider that the varying the period of inactivity
breaks the periodic pulse generation. The network learning
obtains the diversity. By the computer simulation, we showthat the pulse generation pattern becomes the diversity.
Moreover proposed network has a better performance thanthe conventional method.
II. P ROPOSED M ETHOD
In this study, we propose the MLP with pulse glial chain
based on individual inactivity period as shown in Fig. 1. We
connect the glias to the neurons in the hidden-layer. The
glia makes the different network from the neural network.

Firstly, the glia receives the connecting neuron output. If
it is over the excitation threshold of the glia, the glia is
excited. The excited glia generates the pulse. This pulse canhave a negative value and a positive value. It is depended
on the connecting neuron output. After that, the pulse is
input to the connecting neuron threshold. Moreover, the pulse
influences to the neighboring glias. The neighboring glias arealso excited by this pulse independent from the connecting
neuron output. Thus, the pulse is propagated into the glia
network. The pulse gives the energy to the network, because
the glia pulse is independent from the network learning.
Moreover, pulse propagation gives the position relationshipwith each neuron in the hidden-layer. The pulse generation
time is similar each other. In the previous method, we fix
the period of inactivity. The period of inactivity decides thecycle of the pulse generation. Then the pulse generation
often became the periodic. We consider that it reduces thepossibility of escaping out from the local minimum. In the
proposed method, we vary the period of inactivity according
to the glia excitation. When the same glia is continuously
excited by the connecting neuron, the period of inactivity ofthis glia becomes a short. The glia obtains the different period
of inactivity each other with time. Thus, this glia exits theperiodic pulse generation because the neighboring gila does
not finish the period of inactivity when this glia finishes theperiod of inactivity.

the proposed method, the length of the period of inactivity
is varied according to the pulse generation. If the glia iscontinuously excited by the connecting neuron output, thelength of the period of inactivity becomes a short. Moreover,
if the glia is excited by the neighboring glia pulse, the period
of inactivity of this glia returns to original time length of theperiod of inactivity. Figure 2 shows the two different pulse
generation. In the upper figure, the pulse generation cycle
becomes short with time. The bottom figure has periodic
pulse generation. In the glia network, the both glias exist
at one time. Thereby, we consider that the pulse generation
pattern is dynamically changed in the proposed method.

Fig. 2. Varying period of inactivity. (a) The length of the period of inactivity
becomes short with time. (b) Periodic pulse generation.

NeuronB. Pulse propagation…
Glia
Fig. 1.

MLP with pulse glial chain based on individual inactivity period.

A. Glia response
The glia has two different states which are the positive
response and the negative response. We define the output
function as the positive response of the glia in Eq. (1).
𝜓𝑖 (𝑡 + 1) ={ 1, {(𝜃 < 𝑦 ∪ 𝜓𝑛𝑖𝛾𝜓𝑖 (𝑡), 𝑒𝑙𝑠𝑒,

𝑖+1,𝑖−1 (𝑡 − 𝑖 ∗ 𝐷) = 1)

∩ (𝜏𝑖 ≥ 𝜃𝑔𝑖 )} , (1)where 𝜓 is an output of a glia, 𝑖 is a position of the glia, 𝜃𝑛
is a glia threshold of excitation, 𝑦 is an output of a connected
neuron, 𝐷 is a delay time of a glial effect, 𝜏 is local timeof the glia during a period of inactivity, 𝜃𝑔 is a length ofthe period of inactivity, 𝛾 is an attenuated parameter. In

1639

Figure 3 shows an example of the pulse generation anda propagation. In this figure, some glias are excited andpulse generates. If the glia receives the large output of theconnecting neuron, this glia generates the positive pulse. If
the glia receives the small output of the connecting neuron,
this glia generates the negative pulse. The red part shows thenegative value pulse, the blue part shows the positive value
pulse. After that this pulses are propagated to the other glias.
Both pulse generations are similar pattern at first. In the caseof (a), we can observe a small change of the pulse generation
pattern. The pulse generation pattern is fixed with time. On
the other hand, the pulse generation pattern (b) piecemeal
varies from (a). Moreover, the pulse generation pattern (b)varies for a long time than (a). From the figure, the proposed
network breaks the periodic pulse generation and makes thediversity.
C. Updating rule of neuron
The neuron has multi-inputs and single output. We canchange the neuron output by the tuning the weights ofconnections. The standard updating rule of the neuron isPosition of the neuron from 1st to 40thTime(5) The MLP with pulse glial chain based on individual
inactivity period (The period of inactivity is varied
according to the pulse generations.).
The network of (1) does not have the external unit, thus thisnetwork is often falls into local minimum. The network of(2) noise has an uniformed random noise. The network of(3) has same period of inactivity in every glia. In the (4),every glia has different the length of the period of inactivity
which is decided at random. Every MLP has same number ofneurons and layers. The MLP is composed of 2-40-1 neurons.
We obtain the experimental result from 100 trials. Every trial
has different initial conditions. One trial has 50000 iterations.
We use Mean Square Error (MSE) for the error function. The
MSE is described by Eq. (5).

(a) Previous pulse generation (b) Proposed pulse generation

Fig. 3. Pulse generation and propagation. (a) The pulses are generated by
the previous glia network. (b) The pulses are generated by the proposed glia
network.

defined by Eq. (2).

⎛𝑦𝑖 (𝑡 + 1) = 𝑓 ⎝𝑛∑⎞𝑤𝑖𝑗 (𝑡)𝑥𝑗 (𝑡) − 𝜃𝑖 (𝑡)⎠ ,

(2)

𝑗=1

𝑁

𝑀 𝑆𝐸 =1 ∑(𝑇𝑛 − 𝑂𝑛 )2 ,𝑁 𝑛=1

where 𝑁 is a number of learning data, 𝑇 is a target value,
and 𝑂 is an output of MLP. We obtain results which are
an average error, a minimum error, a maximum error, and astandard deviation of the results.
A. Simulation task

where 𝑦 is an output of the neuron, 𝑤 is a weight ofconnection, 𝑥 is an input of the neuron, and 𝜃 is a thresholdof neuron. In this equation, the weight of connection and thethreshold of the neuron are learned by BP algorithm. Thus,
the neuron output is depended on the BP learning. Next, weshow a proposed updating rule of the neuron. We add theglial pulse to the threshold of neuron. We use this updating
rule to the neurons in the hidden layer. It is described by
Eq. (3).
⎛⎞𝑛∑𝑤𝑖𝑗 (𝑡)𝑥𝑗 (𝑡) − 𝜃𝑖 (𝑡) + 𝛼𝜓𝑖 (𝑡)⎠ , (3)𝑦𝑖 (𝑡 + 1) = 𝑓 ⎝We use a Two-Spiral Problem (TSP) for the simulation
task which is shown in Fig. 4. The TSP is famous task for theartificial neural network [7][8]. It has high nonlinearity. Thus,
the standard MLP often falls into the local minimum. In thistask, we input the spiral coordinates to the MLP (shown as
Fig. 4). After that the MLP learns the classification of the
spiral. We change the number of spiral points (98 and 130
points) and obtain the result from two simulations. Figure 5
shows that a classification of 𝑥 − 𝑦 plane which is obtained
from a norm of each coordinate.

𝑗=1

where 𝛼 is a weight of the glial effect. We can change theglial effect by change of 𝛼. In this equation, the weight ofconnection and the threshold are changed by BP algorithm assame as the standard updating rule of the neuron. However,
the glial effect is not changed. It is updated by Eq. (1).
Equations (2) and (3) are used a sigmoidal function to an
activating function which is described by Eq. (4).
𝑓 (𝑎)

(5)

=

1
1 + 𝑒−𝑎Fig. 4.

Two-Spiral Problem.

(4)

where 𝑎 is an inner state.
III. S IMULATIONS
We compare five kinds of the MLPs;(1) The standard MLP.
(2) The MLP with random noise.
(3) The MLP with pulse glial chain.
(4) The MLP with pulse glial chain based on individual
inactivity period (The period of inactivity is random.).

1640

Fig. 5.

Classification of Two-Spiral Problem in 𝑥 − 𝑦 field.

B. Simulation results
1) The number of spirals are 98: Firstly, we use the 98
spiral points to the learning of the MLP. The learning performance means the fitting between the output of the MLP and
the supervised classification. Table I shows the experimental
result of the learning performance. Every method improves
the performance than the standard MLP. From this result,
we can see that the proposed MLP has a three times better
performance than the MLP with pulse glial chain. The MLP
with pulse glial chain and proposed MLP have the better
performance than the MLP with random noise, thus the pulse
is efficient to the MLP learning. Moreover, we consider thatthe pulse generation pattern is important to the MLP learning.

have crack at the periphery of (𝑥, 𝑦) = (1, 0.5). The MLP
with pulse glial chain can draw the two spirals, however it
also has some errors at the periphery of (𝑥, 𝑦) = (1, 0.5).
A border value of the two spirals becomes about (𝑥, 𝑦) =(1, 0.7). On the other hand, our proposed MLP can obtainthe two spirals in the field, moreover it does not have thelarge error in every area.

TABLE IL EARNING PERFORMANCE OF SPIRAL OF 98 POINTS .

(1)
(2)
(3)
(4)
(5)

Average
0.04153
0.037110.015310.017910.00444

Minimum
0.000170.00006
0.00009
0.000160.00016

Maximum
0.183870.17352
0.06157
0.18380
0.04151

TABLE IIC LASSIFICATION PERFORMANCE OF SPIRAL OF 98 POINTS .
Average
0.150290.13966
0.109800.11647
0.09565

Minimum
0.080850.08083
0.064080.07176
0.06188

Maximum
0.211270.203780.15069
0.17159
0.17970(b) MLP with random noise(c) MLP with pulse glial chain.

(d) Proposed MLP (random).

Std. Dev.
0.02637
0.02946
0.016360.024150.00956

Table II shows the classification performance. We input
the unlearning coordinates to the MLP which finishes thelearning. After that we obtain the output of the MLP in
correspondent of the input coordinates. We compare the trueclassification and the output of the MLP. The true classification is obtained from norm between the input classificationand the learning spiral coordinate. The trend of the results
is similar to the learning performance. We can see that the
proposed MLP is only under 0.1 in the average of error.
In the learning performance, the proposed MLP has a highability. In general, the MLP becomes the over fitting when it
has too much learning, because the MLP falls the deep local
optimum solution. However the proposed MLP can classify
the unknown data than the others, it means that the proposed
MLP has a high generalization capability. From this result,
our method can find a better solution. Moreover it can searcha wide range of a solution space.

(1)
(2)
(3)
(4)
(5)

(a) Standard MLP.

Std. Dev.
0.02434
0.02879
0.019020.023100.01773

Figure 6 shows the classification of unknown coordinates
when the MLP learns the 98 spiral points. We obtain thesefigures from the near average result in Table II. The standard
MLP, the MLP with random noise, and the MLP with pulse
glial chain based on individual inactivity period (The period
of inactivity is random.) cannot draw the spirals. These MLPs

1641(e) Proposed MLP.

Fig. 6. Classification of two spirals of 98 points for unknown coordinates.

2) The number of spirals are 130: Secondly, we show
the learning performance of the spirals of 130 points. Of
course, the TSP becomes difficult by increasing the numberof the spiral points. In this case, the number of turns is also
improves, thus this task has stronger nonlinearity than theprevious task. The statistic result shows in Table III. We cansee that the standard MLP often traps into the local minimum.
Thereby, the average of error is the worst of all. The result of
the MLP with random noise is similar to the standard MLP.
From this result, the uniformed random noise is not efficientto the TSP. Other three MLPs improve the performance fromthe result of the standard MLP. Especially, the MLP withpulse glial chain and the proposed MLP have a good learning
performance. Moreover, the maximum error of the proposed
MLP is the best of all. From this result, we can say thatthe proposed MLP has a high ability for escaping out fromthe local minimum. Thereby, our proposed MLP reduces aninitial valued dependence. It means that we can stably obtain

0.35

the better result.
TABLE IIIL EARNING PERFORMANCE OF SPIRAL OF 130 POINTS .
Minimum
0.008310.000470.000670.001340.00052

Maximum
0.23857
0.24278
0.116640.144810.04851

0.25

Std. Dev.
0.055540.057420.02226
0.03608
0.01313

MSE

(1)
(2)
(3)
(4)
(5)

Average
0.12269
0.108470.019900.05546
0.01414

0.3

䠄2䠅

0.2

0.15

Next, we show the classification performance of the MLPs.
The classification results show in Table IV. The trend of thesimulation results is similar to the learning performance. The
standard MLP and the MLP with random noise are worse
results. The classification performance of the proposed MLP
is the best of all in every index.

00Minimum
0.10565
0.10460
0.08027
0.09368
0.06876

Maximum
0.294770.33065
0.196390.243280.19142

10000
Fig. 7.

Std. Dev.
0.03858
0.04434
0.02625
0.029480.02473

20000

30000

40000

50000

Iteration
Learning curve.

pattern. Actually, the MLP with pulse glial chain early fix thepulse generation pattern, thereby the error reduction becomes
gradual with temporal progress.
0.3

Figure 7 shows learning curves of each MLP. The error
reduction of the standard MLP converges at 25000. It istrapped into the local minimum. The convergence of the error
in the MLP with random noise is a slower than the others.
However, it reduces the error than the standard MLP. The
uniformed random noise has a small efficiency to the learningof the MLP. On the others, these curves have a oscillation
during the iterations. Moreover the performance of the error
reduction improves. The pulse locally gives the large energy
to the network. The pulse helps escaping out from the local
minimum. The glia has the period of inactivity. During theperiod of inactivity, the glia does not generate the pulse again.
Thereby, the MLP can search the better solution during theperiod of inactivity. The error reduction of the proposed MLP
is earlier than the others. Thus, the pulse generation patterninfluences the learning of the MLP.
By using this result, we compare the proposed MLP with
the MLP with pulse glial chain. The proposed MLP has
a better performance than the MLP with pulse glial chain,
however a difference of superiority in the statistic result is
not observed. Here, we show the error reduction curves of the
proposed MLP and the MLP with pulse glial chain (shownas Fig. 8) which is obtained from an average error at each
iteration. We can see that the error of the proposed MLP
rapidly decreases from a start of the learning, moreover theerror converges the learning earlier than that of the MLP
with pulse glial chain. We consider that it is an influence ofchanging period of inactivity, because the proposed MLP canvary the pulse generation pattern by changing the period ofinactivity. We consider that the influence of the pulse glial
chain becomes small by convergence of the pulse generation

1642

Previous MLP

0.25

Proposed MLP
0.2

MSE

Average0.21782
0.19278
0.12538
0.153340.11857

䠄3䠅

䠄5䠅

0.05

TABLE IVC LASSIFICATION PERFORMANCE OF SPIRAL OF 130 POINTS .

(1)
(2)
(3)
(4)
(5)

䠄1䠅

䠄4䠅

0.1

0.15
0.1
0.05
0
0

10000

20000

30000

Iteration

40000

50000

Fig. 8. Comparison of the convergence of the proposed MLP and the MLP
with pulse glial chain.

Finally, we show the classification image of the TSP as
shown in Fig. 9. We obtain the classification image fromaverage result in Table IV. The standard MLP cannot draw
the spirals, thus the solving ability of the standard MLP is
unsatisfactory for the TSP. We can observe the outside circleof the spiral in the MLP with random noise however it has
any cracks. The MLP with pulse glial chain can classifier the
spirals. The MLP with pulse glial chain based on individual
inactivity period (the period of inactivity is random) has two
cracks. This model is similar to the proposed MLP however
its performance is worse in every result. The proposed MLP
can also separate the two spirals. Moreover, the outside curve

is better than the MLP with pulse glial chain. Actually, theMLP with pulse glial chain has error near coordinates (0.0,0.5).

ACKNOWLEDGMENT
This work was partly supported by MEXT/JSPS Grant-inAid for JSPS Fellows (24⋅10018).
R EFERENCES(a) Standard MLP.

(b) MLP with random noise(c) MLP with pulse glial chain.

(d) Proposed MLP (random).

[1] P.G. Haydon, “Glia: Listening and Talking to the Synapse,” Nature
Reviews Neuroscience, vol. 2, pp. 844-847, 2001.
[2] S. Koizumi, M. Tsuda, Y. Shigemoto-Nogami and K. Inoue, “Dynamic
Inhibition of Excitatory Synaptic Transmission by Astrocyte-Derived
ATP in Hippocampal Cultures,” Proc. National Academy of Scienceof U.S.A, vol. 100, pp. 11023-11028, 2003.
[3] S. Ozawa, “Role of Glutamate Transporters in Excitatory Synapses
in Cerebellar Purkinje Cells,” Brain and Nerve, vol. 59, pp. 669-676,
2007.
[4] G. Perea and A. Araque, “Glial Calcium Signaling and Neuro-Glia
communication,” Cell Calcium, vol. 38, 375-382, 2005.
[5] D.E. Rumelhart, G.E. Hinton and R.J. Williams, “Learning Representations by Back-Propagating Errors,” Nature, vol. 323-9, pp. 533-536,
1986.
[6] C. Ikuta, Y. Uwate, and Y. Nishio, “Multi-Layer Perceptron with Positive and NegativePulse Glial Chain for Solving Two-Spirals Problem,”Proc. IJCNN’12, pp.2590-2595, Jun. 2012.
[7] J.R. Alvarez-Sanchez, “Injecting knowledge into the Solution of the
Two-Spiral Problem,” Neural Computing & Applications, vol. 8,
pp. 265-272, 1999.
[8] H. Sasaki, T. Shiraishi and S. Morishita, “High precision learning forneural networks by dynamic modification of their network structure,”
Dynamics & Design Conference, pp. 411-1–411-6, 2004.

(e) Proposed MLP.

Fig. 9. Classification of two spirals of 130 points for unknown coordinates.

IV. C ONCLUSIONSIn this study, we have proposed the MLP with pulse glial
chain based on individual inactivity period. We connect theglia to the neuron in the hidden-layer. The glia receives
the connecting neuron output. The glia generates the pulse
when the neuron output is over the excitation thresholdof the glia. This pulse is input to the connecting neuron
threshold and moreover it is propagated to the neighboring
glias. In this method, the period of inactivity is varied
according to the pulse generation time. If the pulse generation
continuously occurs by the connecting neuron output, theperiod of inactivity becomes short. By this influence, thepulse generation pattern is dynamically changed because theperiod of inactivity of the glia is different each other. We
consider that the glia pulse improves the MLP performance.
Actually, we confirm that the proposed MLP has a better
performance than the conventional MLP by the computer
simulation.

1643