Proceedings of International Joint Conference on Neural Networks, San Jose, California, USA, July 31 – August 5, 2011

Performance and Features ofMulti-Layer Perceptron with Impulse Glial Network
Chihiro Ikuta, Yoko Uwate and Yoshifumi Nishio

Abstract— We have proposed the glial network which was
inspired from the feature of the brain. The glial network is
composed by glias connecting each other. All glias generate
oscillations and these oscillations propagate in the glial network.
We conﬁrmed that the glial network improved the learning
performance of the Multi-Layer Perceptron (MLP).
In this article, we investigate the MLP with the impulse
glial network. The glias generate only impulse output, however
they make the complex output by correlating with each other.
We research the proposed networks’ parameter dependency.
Moreover, we show that the proposed network possess better
learning performance and better generalization capability thanthe conventional MLPs.

I. I NTRODUCTION
URRENTLY, various artiﬁcial neural network models
are proposed. They mostly imitate a biological brain
system and are useful to solve difﬁcult tasks. The MultiLayer Perceptron (MLP) is one of feed forward neuralnetworks and is useful to perform several tasks, for example,
pattern recognition, pattern classiﬁcation, data mining and soon. Back Propagation (BP) algorithm is a learning algorithm
for the MLP using the steepest decent method [1]. The
MLP can solve the linearly–inseparable problem by using
this learning algorithm. However, when we use this learningalgorithm, we must be careful to the local minimum problem.
The MLP can not often ﬁnd the global solution from theinﬂuence of this problem.
We have proposed the glial networks to improve theperformance of MLP [2][3]. The glial network was inspired
from the feature of the glias which are existing in thebrain. The glias were not attract researchers’ attentions fora long time, because the glias have been believed not to use
electrical signals. Recently, some researchers discovered thatthe glias transmit signals by using ions [4]. In the brain,
the glias inﬂuence each other and the ions affect neurons’
thresholds [5][6]. We tried to exploit the glia network’s
behavior such that another network located close to theneural network helps the functions of the neural networks.
In [2], we proposed the MLP with the glial network whose
glias generated independent oscillations and these oscillations propagated neurons and other glias. We conﬁrmed bycomputer simulations that the glial network improved thelearning performance of the MLP by connecting the neurons
more effectively than the conventional networks.

C

Chihiro Ikuta, Yoko Uwate and Yoshifumi Nisho are with the Department of Electrical and Electronic Engineering, The Tokushima University, 2-1 Minami-Josanjima, Tokushima, Japan, (email: {ikuta, uwate,nishio}@ee.tokushima-u.ac.jp).

978-1-4244-9636-5/11/$26.00 ©2011 IEEE

In this paper, we propose a MLP with the impulse glial
network. In the biological glias, the glias generate Ca2+
density wave which is known to like impulse [7]. If the glias
are excited, they rapidly change the Ca2+ density in the brain.
After that, the Ca2+ density return to a normal state with time[8][9]. In order to conﬁrm the relationship between glias andneurons, we investigate the impulse glial network. In our
network, all glias generate impulses when the glia is excitedby the connecting neuron. The glia connects the neuron andwatch to neuron’s output. If the neuron’s output becomes tohigh, the glia generates an impulse. After that, this impulse
is propagating in the glial network, this glia can not generate
the impulse during constant time like biological neuron’s
refractory period. The glias generate simplex impulse output,
however, they have the glial network. They can make thecomplex output by correlating each other. We consider thatall glias generate the impulses at random times each other,
the glias’ effects give good inﬂuence to the MLP. We show
the performance and features of proposed MLP with theimpulse glial network by the computer simulations.
II. MLP WITH I MPULSE G LIAL N ETWORKThe MLP is the most famous feed forward neural network.
Several methods using the MLP have been proposed forsolving many kinds of tasks. This network has some neuron
layers and the weights between the layers are learned by
the BP algorithm. In this study, we use the MLP with threelayers (4-10-1) and a glial network connected to the hidden
layer. Figure 1 shows an example of the MLP with the glial
network.

Fig. 1.

MLP with impulse glial network.

2536

A. Neuron Updating Rule
The standard neuron updating rule is given by Eq. (1).
⎛⎞𝑛∑𝑤𝑖𝑗 (𝑡)𝑥𝑗 (𝑡) − 𝜃𝑖 (𝑡)⎠ ,(1)𝑥𝑖 (𝑡 + 1) = 𝑓 ⎝𝑗=1

where 𝑥 : input or output, 𝑤 : weight parameter, 𝜃 : threshold
and 𝑓 : output function. The parameters 𝑤 and 𝜃 are learned
by using the BP algorithm.
The updating rule of the hidden layer’s neurons of the
proposed neural network with the glial network is modiﬁed
as Eq. (2).
⎛⎞𝑛∑𝑤𝑖𝑗 (𝑡)𝑥𝑗 (𝑡) − 𝜃𝑖 (𝑡) + 𝛼Ψ𝑖 (𝑡)⎠ , (2)𝑥𝑖 (𝑡 + 1) = 𝑓 ⎝𝑗=1

Fig. 2.

where Ψ : output of the glias, 𝛼 : weight of glia outputs. We
use the sigmoid function for the output 𝑓 as Eq. (3).
𝑓 (𝑎)

=

1
1 + 𝑒−𝑎Outputs of the glias in three different positions.

(3)

B. Impulse Glial Network
In the biological neural network, it is known that the glias
affect the neighboring neurons over a wide range by making
their outputs to be propagated in the network.
The output of the glias can be given as Eq. (4).
𝑚∑Ψ𝑖 (𝑡) =𝛽 ∣𝑘∣ 𝜓𝑖+𝑘 (𝑡 − ∣𝑘∣),

(4)

𝑘=−𝑚

Fig. 3.

Outputs of the impulse glial network as ﬁrst glia and second glia.

Fig. 4.

Outputs of the impulse glial network as ﬁrst glia and sixth glia.

where 𝜓 is the glia making impulse, 𝛽 denotes attenuation
parameter and 𝑚 is propagating range of glia’s impulse. We
deﬁned the glia’s impulse, it is given by Eq. (5).
𝜓𝑖 (𝑡 + 1)
𝜓𝑖 (𝑡 + 1)==

1, (𝜃𝑛 < 𝑂𝑖 ) ∩ (𝜃𝑔 > 𝜓𝑖 (𝑡))𝛾𝜓𝑖 (𝑡), 𝑒𝑙𝑠𝑒,(5)

where 𝜃𝑛 is a threshold of existing the glia, 𝑂 is an output
of each neuron, 𝜃𝑔 is a threshold of glia’s refractory periodand 𝛾 is an attenuated parameter of the glial output. The
glia generates the impulse output as exciting by neuron’s
output. However, if the neuron’s output is not over thethreshold or into the refractory period, the glia can notgenerate the impulse and this glia’s output is attenuated by 𝛽.
In the Fig. 2, we show the generating impulses by the threedifferent positions’ glias. 𝜓1 (𝑡) and 𝜓2 (𝑡) are generated byneighborhood glias. 𝜓6 (𝑡) is generated by the far glia from
𝜓1 (𝑡).
We consider that each glia can make only impulse, however the glial network can make complex oscillations by
the glias connecting each other. One example of the glial
network’s oscillation is shown in Figs. 3 and 4. Ψ1 (𝑡) issimilar to Ψ2 (𝑡), because they are correlation each other by
the glial network. Between Ψ1 (𝑡) and Ψ6 (𝑡), their correlation
is weak, thus, their oscillations become different patterns.

C. Back Propagation
The error of the MLP propagates backward in the feed
forward neural network. The BP algorithm changes thevalues of the weights between the neurons to obtain smaller
error than before. The error of the network is given by
Eq. (6).
𝑛𝐸=

1∑(𝑡𝑖 − 𝑂𝑖 )2 ,
2 𝑖=1

(6)

2537

where 𝐸: error value and 𝑡: target value.
III. S IMULATION R ESULTS
In this section, we show the performance of the proposed
MLP with the impulse glial network by learning two kindsof chaotic time series. We use the skew tent map as different
𝐴 to generate chaotic time series. The skew tent map isformulated by Eq. (7).
{ 2𝜙(𝑡)+1−𝐴 (−1 ≤ 𝜙(𝑡) ≤ 𝐴)
𝜙𝑖 (𝑡 + 1) =1+𝐴

−2𝜙(𝑡)+1+𝐴
1−𝐴(7)(𝐴 < 𝜙(𝑡) ≤ 1)

Figure 5 is skew tent map which is given by Eq. (7).

(a) Step 1.

(b) Step 2.

Fig. 7.

(c) Step 3.

Learning procedure.

We compare the performances of four different MLPs,
which are the conventional MLP, the MLP with the impulse
glial network, the MLP with the glial network and the MLP
with random noise. We use Mean Square Error (MSE) for
evaluate these performances.
𝑁

𝑀 𝑆𝐸 =

1 ∑(𝑡𝑖 − 𝑂𝑖 )2 ,
𝑁 𝑖=1

(8)

where N is the number of trials.
A. Parameter Dependency

Fig. 5.

Skew tent map (𝐴 = −0.10, 0.10).

The skew tent map generates chaotic time series between−1 and 1, thus, we transform chaotic time series whichbecome between 0 and 1. The MLP learns to two different
chaotic time series of classiﬁcation. If the MLP takes 𝐴 =−0.10 chaotic time series, the MLP learns to 1 and if the
MLP takes 𝐴 = 0.10 chaotic time series, the MLP learns to
0. Figure 6 shows example of learning chaotic time series.

Fig. 6.

Chaotic time series (𝐴 = −0.1, 0.1).

The BP learning for the MLPs is carried out by giving four
successive points of the chaotic time series as an input andthe following 0 or 1 as an output. The learning is repeated for
200 different sets which are included two different chaotic
time series like Fig. 7.

In this simulations, we conﬁrm the feature of the MLP
with the impulse glial network by researching parameters
dependency. In each simulation, we obtain the average error
from the 100 trials.
1) 𝛼 dependency: First, we show that the learning performance of the MLP with impulse glial network by using
different 𝛼 in Eq. (5). 𝛼 is the amplitude of the oscillations,and inﬂuences directly to the MLP learning. We change theamount of the 𝛼 from 0.1 to 0.9 and obtain the MSE as
each 𝛼. For comparing the results, we show the results of
the MLP with the glial network and the MLP with random
noise.
Figure 8 is amount of error as each 𝛼. The error distribution of the MLP with random noise become ﬂat. From thisresult, we can say that the 𝛼 dependency of the MLP withrandom noise is weaker than the others. The error of the MLP
with the glial network becomes small around 𝛼 = 0.15, andbecome large value in larger 𝛼. In the case of the MLP with
the impulse glial network, the error become small value inlarger 𝛼. We consider that the MLP with the impulse glial
network can use large value of 𝛼 because the MLP has theperiod of inactivity. The MLP can learn true target during
the period of inactivity if the MLP is given the large input
by the impulse glial network.
2) 𝜃 dependency: Next, we show the results as two kindsof thresholds. Figure 9 is the result of changing threshold
which is limit to be excited glia as Eq. (5). If this threshold isincreased, the glias need large amount of the neurons’ output
for generating the impulse outputs. In this ﬁgure, we change
the threshold from 0.50 to 0.95 because we consider that theglia become exciting by positive output of the neuron. We
use sigmoidal function for the neuron’s output function, thus,𝜃𝑛 = 0.50 means that the neurons have positive inner state.
In Fig. 9, the errors become small on the whole. The results
between 0.60 and 0.90 especially become small error. We
consider that the MLP with the impulse glial network does

2538

Fig. 8.

𝛼 dependency of the MLPs.
Fig. 10.

not have highly dependency from this result, however, we
can see that the result become bad around the 0.05. Thisbecause of the glias are excited by the small outputs of theneurons and the glias generate positive value output. Thus,
the entire neurons’ work become positive direction.

Fig. 9.

𝜃𝑔 dependency of the MLP with the impulse glial network.

The graph inclination is that when 𝛾 is lager, the error issmall in Fig. 11. As 𝛾 = 0.1, the performance of the MLP
with the impulse glial network is the worst. This is becauseof the glias’ output is rapidly decreased by 𝛾. Thereby, we
consider that the glias can not inﬂuence to the MLP learning.
We hold on amount of 𝛽 as Eq. (4), the glia generating
impulse affect around glia, however, it can not affect by itself.

𝜃𝑛 dependency of the MLP with the impulse glial network.

Figure 10 is error as the thresholds of glia’s refractory
period. 𝜃𝑔 operates time length of the period of inactivity.
If the output of each glia must become smaller than thisthreshold, the glias can not generate next impulse output. The
glias’ outputs are decreased value in an exponential fashion.
Thereby, we change the threshold from 10−5 to 10−1 in an
exponential fashion.
This result shows that the error as large 𝜃𝑛 is larger
than the error as small 𝜃𝑛 . In larger 𝜃𝑛 , the glias generate
the impulse output at short intervals. We consider that theperformance of the MLP become well when the glias’
outputs are like more impulse from this ﬁgure. In fact, theglia’s exciting intervals are longer than neurons’ work in the
biological brain.
3) 𝛾 dependency: Finally, we show 𝛾 density of the MLP
with the impulse glial network as Eq. (5). 𝛾 is the attenuatedparameter of the glia’s output. If 𝛾 becomes large, to beattenuated value of the glia’s output become small it the one
time.

Fig. 11.

𝛾 dependency of the MLP with the impulse glial network.

Figure 12 is a graph of the relationship between 𝛾 and 𝛽.
Both 𝛾 and 𝛽 are attenuated parameters. From this ﬁgure, we
can see that the error is small value as 𝛽 = 0.4 or 𝛽 = 0.6
which are independent from the value of 𝛾. When both 𝛾
and 𝛽 are small value, the errors are large. In this case, theglias’ output reduce rapidly by these two parameters. We
consider that the glias must keep up the output value forcertain time. And also, when the ratio of 𝛾 with 𝛽 is high,the error become large.
B. Learning Performance
In this paragraph, we investigate the learning performanceof the MLPs by calculating the time evolution of the errors.
The number of the trials is 100 and the MLPs learn 25000

2539

0.03
0.025

MSE

0.02
0.015
0.01
0.005
0
0.8
0.6
γ

0.4
0.2
0.1

Fig. 12.

0.2

0.3

0.4

0.5

0.6

0.7

0.8

β

Fig. 13.

Error curve of four different MLPs.

𝛾–𝛽 characteristic.

C. Generalization Capability
times during one trial. We calculate the average of error(Avg. Err.), the minimum error (Min.), the maximum error(Max.) and the standard deviation (St. Dev.). Table I shows
the obtained results.
TABLE IL EARNING PERFORMANCE .
Conventional
Glia
Impulse Glia
Noise

Avg. Err.
0.03510.00740.00280.0111Min.
0.00110.00160.00040.0009Max.
0.11790.06440.06100.1098

Finally, in order to evaluate the generalization capabilitiesof the MLPs, we give the MLPs unknown time series
generated by the same skew tent map after the learning andevaluate the errors between the outputs and the correct value.
We use ten different time series including 200 points for thetest data. The learning data and one example of the test data
are shown in Fig. 14.

St. Dev.
0.04340.01250.00620.0190In the average of error, we can see that the conventional
method is the worst. The MLP with the impulse glial
network and the MLP with the glial network reduce more
the maximum error than the conventional MLP and the MLP
with random noise. We consider that the MLP with the glial
networks could escape out from the largest local minimum.
The MLP with the impulse glial network is better than the
MLP with the glial network. We consider that the MLP with
the impulse glial network has the period of inactivity. In the
glial network, all glias make random value output, thus, the
MLP is possible learned to false value over long periods oftime. The MLP with the impulse glial network can learn totrue value during the period of inactivity. Because the if theglias make large impulse, the output is decreased and is not
generated impulse during this period.
Figure 13 shows an example of the learning curves. The
error of the conventional MLP converges constant value andis not improved any more. The MLP with the impulse glial
network reduced the error earliest of all. This because ofneighborhood neurons are inputted similar oscillations by theimpulse glial network. We assume that the similar oscillations
give good inﬂuence to the MLP learning.

Fig. 14.

Learning data and test data.

This result is summarized in Tab. II. From Tab II, we
can see that the MLP with the impulse glial network is thebest as average of error. However, this network’s maximum
error is the worst, this is because of this network having theoscillations by the glial network, thus this network took badstates at the ﬁnishing learning.
TABLE IIG ENERALIZATION C APABILITY.
Conventional
Glia
Impulse Glia
Noise

Avg. Err.
0.0496
0.03450.03260.0443Min.
0.00060.00600.00040.0008Max.
0.16840.13800.21560.1744

St. Dev.
0.04680.02160.02320.04192540

IV. C ONCLUSION
In this article, we have proposed the MLP with the impulse
glial network. The glias generate the impulse output by
the MLP’s output, moreover the outputs are propagating toother glias, thus the glial network can generate complex
oscillations. We could see that the glias’ outputs were similar
patterns as local positions in the impulse glial network.
By computer simulations, we showed that the changing
performance of MLP by the each parameter as the impulse
glial network and found the good parameters for learning
MLP. Moreover, we conﬁrmed that the proposed MLPs with
the impulse glial networks is better learning performanceand better generalization capability than the conventional networks. The impulse glial network has the period of inactivity,
thus, this network can have large amplitude oscillations. We
consider that this behavior gave good inﬂuence to the MLP
learning.
In the Future works, we will particularly research about
outputs of the impulse glial network. Neighborhood glias
generating outputs looks like synchronizing each other. We
assume that the correlation of the glias is important for the
MLP learning.
R EFERENCES[1] D.E. Rumelhart, G.E. Hinton and R.J. Williams, “Learning Representations by Back-Propagating Errors,” Nature, vol. 323-9, pp. 533-536,
1986.
[2] C. Ikuta, Y. Uwate and Y. Nishio, “Chaos Glial Network Connectedto Multi-Layer Perceptron for Solving Two-Spiral Problem,” Proc.
ISCAS’10, pp. 1360-1363, May 2010.
[3] C. Ikuta, Y. Uwate and Y. Nishio, “Multi-Layer Perceptron with Impulse
Glial Network,” Proc. NCN’10, pp.9-11, Dec. 2010.
[4] P.G. Haydon, “Glia: Listening and Talking to the Synapse,” Nature
Reviews Neuroscience, vol. 2, pp. 844-847, 2001.
[5] S. Koizumi, M. Tsuda, Y. Shigemoto-Nogami and K. Inoue, “Dynamic
Inhibition of Excitatory Synaptic Transmission by Astrocyte-Derived
ATP in Hippocampal Cultures,” Proc. National Academy of Science of
U.S.A, vol. 100, pp. 11023-11028, 2003.
[6] S. Ozawa, “Role of Glutamate Transporters in Excitatory Synapses inCerebellar Purkinje Cells,” Brain and Nerve, vol. 59, pp. 669-676, 2007.
[7] G. Perea and A. Araque, “Glial Calcium Signaling and Neuro-Glia
communication,” Cell Calcium, vol. 38, 375-382, 2005.
[8] S. Kriegler and S.Y. Chiu, “Calcium Signaling of Glial Cells along
Mammalian Axons,” The Journal of Neuroscience, vol. 13, 4229-4245,
1993.
[9] M.P. Mattoson and S.L. Chan, “Neuronal and Glial Calcium Signalingin Alzheimer’s Disease,” Cell Calcium, vol. 34, 385-397, 2003.

2541