2010 IEEE Workshop on
Nonlinear Circuit Networks, Tokushima, Dec 3-4, 2010

Multi-Layer Perceptron with Impulse Glial Network
Chihiro Ikuta

Yoko Uwate

Yoshifumi Nishio

Dept. of Electrical and Electronic Eng., Dept. of Electrical and Electronic Eng., Dept. of Electrical and Electronic Eng.,
Tokushima University
Tokushima University,
Tokushima University
Email: ikuta@ee.tokushima-u.ac.jp
uwate@ee.tokushima-u.ac.jp
nishio@ee.tokushima-u.ac.jp

Abstract— We have proposed the glial network which wasinspired from the feature of brain. In the glial network, glias
generate independent oscillations and these oscillations propagated neurons and other glias. We confirmed that the glial
network improved the learning performance of the Multi-Layer
Perceptron (MLP)
In this article, we investigate the MLP with the impulse glialnetwork. The glias have only impulse output, however they makethe complex output by correlating with each other. The simulation
result shows that the proposed networks possess better learning
performance than the conventional MLP.

I. I NTRODUCTION
The artificial neural network is a brain’s neurons model
which is variously applied, the neural network was inspired
from behavior of the biological neurons. In the brain, theneural network manages much information by the signaltransmitting of neurons. The Multi-Layer Perceptron (MLP)
is one of feed forward neural networks and is useful toperform several tasks, for example, pattern recognition, patternclassification, data mining and so on. Back Propagation (BP)algorithm is a learning algorithm for the MLP using thesteepest decent method [1].
We have proposed the glial networks to improve the performance of MLP. The glial network was inspired from thefeature of the glias which are existing in the brain. Recently,
some researchers discovered that the glias transmit signals byusing ions [2]. In the brain, the glias influence each otherand the ions affect neurons’ thresholds [3][4]. We tried toexploit the glia network’s behavior such that another network
located close to the neural network helps the functions of
the neural networks. In [5], we proposed the MLP with theglial network whose glias generated independent oscillations
and these oscillations propagated neurons and other glias.
We confirmed by computer simulations that the glial network
improved the learning performance of the MLP by connecting
the neurons more effectively than the conventional networks.
In this paper, we propose a MLP with the impulse glialnetwork. In this network, all glias generate impulses when theglia is excited by the connecting neuron. The glia connects theneuron and watch to neuron’s output. If the neuron’s output
becomes to high, the glia generates an impulse. After that,
this impulse is propagating in the glial network, this glia cannot generate the impulse during constant time like biologicalneuron’s refractory period. We consider that all glias generate
the impulses at random times each other, the glias’ effects give
good influence to the MLP.

II. MLP WITH I MPULSE G LIAL N ETWORKThe MLP is the most famous feed forward neural network.
Several methods using the MLP has been proposed for solving
many kinds of tasks. This network has some neuron layers
and the weights between the layers are learned by the BPalgorithm. In this study, we use the MLP with three layers(4-10-1) and a glial network connected to the hidden layer.
A. Neuron Updating Rule
The standard neuron updating rule is given by Eq. (1).
n∑xi (t + 1) = f wij (t)xj (t) − θi (t) ,(1)j=1where x : input or output, w : weight parameter, θ : threshold
and f : output function. The parameters w and θ are learned
by using the BP algorithm.
The updating rule of the hidden layer’s neurons of theproposed neural network with the glial network is modified
as Eq. (2).
n∑wij (t)xj (t) − θi (t) + αΨi (t) ,xi (t + 1) = f (2)j=1where Ψ : output of the glias, α : weight of glia outputs. We
use the sigmoid function for the output f as Eq. (3).
1f (a) =(3)1 + e−aB. Impulse Glial Network
In the biological neural network, it is known that the glias
affect the neighboring neurons over a wide range by making
their outputs to be propagated in the network. The output ofthe glias can be given as Eq. (4).
m∑Ψi (t) =β |k| ψi+k (t − |k|),

(4)

k=−m

where ψ is the glia making impulse, β denotes attenuation
parameter and m is propagating range of glia’s impulse. We
defined the glia’s impulse, it is given by Eq. (5).
ψi (t + 1)
ψi (t + 1)==1, (θn < Oi ) ∩ (θg > ψi (t))βψi (t), else,(5)

where θn is a threshold of existing the glia, O is an output
of each neuron and θg is threshold of glia’s refractory period.

-9-The glia generates the impulse output as exiting by neuron’soutput. However, if neuron’s output is not over threshold or
into the refractory period, the glia can not generate the impulseand this glia’s output is attenuated by β.
III. S IMULATION R ESULTS(a) Step 1.

In this section, we show the performance of the proposed
MLP with the impulse glial network by learning two kinds ofchaotic time series. We use the skew tent map as different A togenerate chaotic time series. The skew tent map is formulated
by Eq. (6).
{ 2ϕ(t)+1−A (−1 ≤ ϕ(t) ≤ A)
1+A

ϕi (t + 1) =−2ϕ(t)+1+A
1−A(6)(b) Step 2.

Fig. 3.

(c) Step 3.

Learning procedure.

We compare the performances of four different MLPs,
which are the conventional MLP, the MLP with the impulse
glial network, the MLP with the glial network and the MLP
with inputted random noise. We use the following Mean
Square Error (MSE) for evaluate these performances.

(A < ϕ(t) ≤ 1)

1 ∑(ti − Oi )2 ,N i=1
N

M SE =

Figure 1 is skew tent map which is given by Eq. (6).

(7)

where N is the number of trials and t is a target value.
We investigate the learning performance of the MLPs by
calculating the time evolution of the errors. The number of thetrials is 100 and the MLPs learn 25000 times during one trial.
We calculate the average of error (Avg. Err.), the minimumerror (Min.), the maximum error (Max.) and the standard
deviation (St. Dev.). Table I shows the obtained results.
TABLE IL EARNING PERFORMANCE .

Fig. 1.

The skew tent map generates chaotic time series between−1 and 1, thus, we linear transform chaotic time series whichbecome between 0 and 1. The MLP learns to two different
chaotic time series of classification. If the MLP takes A =−0.10 chaotic time series, the MLP learns to 1 and if theMLP takes A = 0.10 chaotic time series, the MLP learns to0. Figure 2 shows example of learning chaotic time series.

Fig. 2.

Conventional
Impulse Glia
Glia
Noise

Skew tent map (A = −0.10, 0.10).

Avg. Err.
0.03510.00530.00740.0111Min.
0.00110.00050.00160.0009Max.
0.11790.06150.0644
0.1098

St. Dev.
0.04340.00680.01250.0190In the average of error, we can see that the conventional
method is the worst. The MLP with the impulse glial networkand the MLP with the glial network reduce more the maximum
error than the conventional MLP and the MLP with inputted
random noise. We consider that the MLP with glial networks
could escape out from the largest local minimum, on the other
hand, they could not escape from it.
Figure 4 shows an example of the learning curves. The
error of the conventional MLP converges constant value and is
not improved any more. The learning curve of the MLP with
the impulse glial network has some small peak, we consider
that the part of glias are synchronized, thus, the impulse glial
network give large influences to the MLP learning, and thisoscillations become a good influence to the MLP.
IV. C ONCLUSION

Chaotic time series (A = −0.1, 0.1).

The BP learning for the MLPs is carried out by giving four
successive points of the chaotic time series as an input andthe following 0 or 1 as an output. The learning is repeatedfor 200 different sets which are included two different chaotic
time series like Fig. 3.

In this article, we have proposed the MLP with the impulse
glial network. The glias generate impulse output by the MLP’soutput, moreover the outputs are propagating to other glias,
thus the glial network can generate complex oscillations. We
could see that the glias’ outputs were sometimes synchronized
as local positions in the impulse glial network.

- 10 -

Fig. 4.

Error curve of four different MLPs.

By computer simulations, we confirmed that the proposed
MLPs with the impulse glial networks is better learning performance. The impulse glial network has state of synchronizationand getting out synchronization, thus, the network can makelarge outputs and small outputs. We consider that this behavior
gave good influence to the MLP learning.
R EFERENCES[1] D.E. Rumelhart, G.E. Hinton and R.J. Williams, “Learning Representations by Back-Propagating Errors,” Nature, vol. 323-9, pp. 533-536,
1986.
[2] P.G. Haydon, “Glia: Listening and Talking to the Synapse,” Nature
Reviews Neuroscience, vol. 2, pp. 844-847, 2001.
[3] S. Koizumi, M. Tsuda, Y. Shigemoto-Nogami and K. Inoue, “DynamicInhibition of Excitatory Synaptic Transmission by Astrocyte-Derived ATPin Hippocampal Cultures,” Proc. National Academy of Science of U.S.A,
vol. 100, pp. 11023-11028, 2003.
[4] S. Ozawa, “Role of Glutamate Transporters in Excitatory Synapses inCerebellar Purkinje Cells,” Brain and Nerve, vol. 59, pp. 669-676, 2007.
[5] C. Ikuta, Y. Uwate and Y. Nishio, “Chaos Glial Network Connectedto Multi-Layer Perceptron for Solving Two-Spiral Problem,” Proc. ISCAS’10, pp. 1360-1363, May 2010.

- 11 -