WCCI 2012 IEEE World Congress on Computational Intelligence
June, 10-15, 2012 - Brisbane, Australia

IJCNN

Multi-Layer Perceptron with Positive and Negative
Pulse Glial Chain for Solving Two-Spirals Problem
Chihiro Ikuta

Yoko Uwate

Yoshifumi Nishio

Dept. of Electrical and Electronics Eng., Dept. of Electrical and Electronics Eng., Dept. of Electrical and Electronics Eng.,
Tokushima University
Tokushima University
Tokushima University
2–1 Minami–Josanjima,
2–1 Minami–Josanjima,
2–1 Minami–Josanjima,
Tokushima Japan
Tokushima Japan
Tokushima Japan
Email: ikuta@ee.tokushima-u.ac.jp
Email: uwate@ee.tokushima-u.ac.jp
Email: nishio@ee.tokushima-u.ac.jp

Abstract—A glia is a nervous cell existing in a brain. The
brain is composed of the relationship with glias and neurons.
By an ion concentration, the glia transmits signal to neurons
and neighboring glias. In this study, we propose the MLP withpositive and negative pulse glial chain which is inspired fromfeatures of the biological glia. We add the MLP to the positive andnegative pulse glial chain. In the positive and negative pulse glial
chain, the glias are connected to the neurons one by one. The glia
generates pulse when the glia is excited by the connected neuron’s
output. If the connected neuron has large amount of output, theglia generates positive pulse. Moreover, if the connected neuron
has small amount of output, the glia generates the negative
pulse. The positive and negative pulse are propagated to theconnected neuron and neighboring glias. We consider that thepositive and negative pulse glial chain give the relationships ofposition of neurons in a same layer. By solving a Two-Spirals
Problem (TSP), we confirm that the proposed MLP has bettera learning performance and a generalization capability than theconventional MLP.

I. Introduction
Nervous cells compose a higher brain function. The nervous
cells are divided into two kinds of cells which are neurons
and glias. Many researchers have researched about biologicalfeatures and its applications of the neurons. The neurons
transmit electric signals to each other, its work composes thethinking, memories, and others. The glia had not been noted,
because the activity of the glia in the brain could be not easily
described. Thereby, this cell is known to statistic and support
cell for a long time. However, some researchers discoveredthat the glias transmit signals by change of concentrations ofseveral ions [1]. The glia has many receptors of ions which
are an adenosine triphosphate (ATP), a glutamate acid (Glu),a calcium ion (Ca2+ ), and so on, moreover, this cell generates
Ca2+ concentration wave [2][3]. These ions are important forthe brain works, actually, the neuron uses the ATP and the Glu
in the gap junction [4]-[6]. We have noticed Ca2+ , because
Ca2+ concentration wave influences the membrane potentialof the neurons [7][8]. Moreover, we consider that the featuresof glia can be applied to artificial neural networks.
Various kinds of artificial neural networks and its applications have been proposed. Multi-Layer Perceptron (MLP)
which is proposed by D.E. Rumelhart is one of the artificial
neural networks [9]. The MLP is composed of the layer ofU.S. Government work not protected by U.S. copyrightneurons. We can obtain the expectation relationships between
inputs and outputs by tuning weights of connections. Back
Propagation algorithm (BP) is often used to the tuning algorithm for the weight of connections. The MLP has theconnections and the relationships between neurons in diﬀerent
layers. However, the neurons do not connect in a same layer.
We noticed this fact and consider that we can give therelationships in the same layer by the network of glias.
In this study, we propose a MLP with positive and negative pulse glial chain. The positive and negative pulse glial
chain is inspired from the features of the biological glia.
All glias one-by-one connect to the neurons in hidden layer
and influence each other. When the neuron generates large
output, the connecting glia is excited and generates positive
pulse. When the neuron generates small output, the connecting
glia is excited and generates negative pulse. The excitation
glia generates pulse which influence neuron threshold andthe excitation of neighboring glias. The output of glia is
attenuated in an exponential fashion. Moreover, the glia has aperiod of inactivity. If the neighboring glia or the connected
neuron aﬀect the excitation glia, the glia cannot be excited
again during the period of inactivity. We consider that the
propagation of glias’ pulses give the relationships of neurons inthe same layer. We believe that the relationships of neurons inthe same layer improve the MLP learning performance. By thecomputer simulation, we confirm that the MLP with positive
and negative pulse glial chain has better learning performance
than the conventional MLP.
II. Multi-Layer perceptron with positive and negative pulse
glial chain
The MLP is the most famous feed forward neural network.
It is composed of the layers of neurons. We can change theoutput of the network by tuning the weights of connections
between neurons. We often use the BP algorithm to decideof weights of connections. In the MLP, the neurons connect
with other neurons in other layer. The MLP can be applied to apattern classification, a pattern recognition, a data mining, andso on. However, it does not have relationships between neurons
in the same layer. When we consider a biological system,
the neuron works have existing the position relationship. We

2590

notice a glia and assume that the glia gives the position
relationship to the neurons. We connect the glia to the neurons
in the hidden layer which is shown in Fig. 1.

Fig. 1.

MLP with positive and negative pulse glial chain.

A. Glial pulse chain
The glia is one of nervous cells existing in a brain. For along time, the glia had not been investigated in detail, because
this cell was considered that it could not transmit signals
similar to neurons. However, some researchers discoveredthat the glia transmits signal by several ions concentrations.
The glia has many ion receptors, for example, ATP, Glu,
GABA, Ca2+ , and so on. These ions are used by the signal ofneuron in the gap junction, thereby the glia is known thathas relationships with neuron signals. We notice the Ca2+
concentration, because the glia generate Ca2+ concentration
wave and it is propagated to wide range in the brain. Moreover,
the Ca2+ aﬀects a membrane potential of the neuron.
In this paper, we propose the positive and negative pulse
glial chain which is inspired from the biological features ofglia. The glias are connected to neurons one-by-one. Whenthe connected neuron has a large output, the glia is excitedand generates the positive pulse. When the connected neuron
has a small output, the glia generates the negative pulse.
The excitation glia generates the pulse. This pulse aﬀects
neighboring glias and the threshold of connected neuron. We
show an example of propagation of glial eﬀect in Fig. 2. One
neuron generates large output and excites the connecting glia.
Next, the excitation glia generates pulse. After that, the pulse
aﬀects neighboring glias and the connected neuron. The glia
which the pulse received is excited, and generate pulses. Then
if the first excitation glia receive the large amount output of
neuron, the glia cannot excite again. Because the glia has aperiod of inactivity. The glias repeat these works and propagate
the glial eﬀects.
The glia has two diﬀerent states which are the positive
response and the negative response. We define the output

Fig. 2.

Pulse propagation.

function as the positive response of the glia in Eq. (1).
ψi (t + 1) =
 1, {(θn < yi ∪ ψi+1,i−1 (t − i ∗ D) = 1)
∩ (θg > ψi (t))} , (1)γψi (t), else,
where ψ is an output of a glia, γ is an attenuated parameter,
y is an output of a connected neuron, θn is a glia threshold of
excitation, θg is a period of inactivity, and D is a delay timeof a glial eﬀect. Moreover, we define the output function as
the negative response of the glia in Eq. (2).
ψi (t + 1) =
 −1, {(1 − θn > yi ∪ ψi+1,i−1 (t − i ∗ D) = 1)
∩ (θg > ψi (t))}γψi (t), else,(2)

The glia does not learn, it depend on the output of connected
neurons. However, the neurons are learned by BP algorithm,
thus the generation pattern of glia output can dynamicallychange during the learning. Figure 3 is an example of generation pulses as 10 glias. At first, the 4th glia, the 6th glia, andthe 9th glia are excited by the connected neurons. There pulses

2591

propagate to neighboring glias. According to the passage oftime, the generation pulse pattern is changed. Actually, thegeneration pulse pattern change from (a) to (b). Because theoutputs of neurons are changed by the learning. We can saythat the glial eﬀect change the neurons threshold, thereby theglias influence the leaning of the MLP.

same as the standard updating rule of the neuron. However,
the glial eﬀect is not changed. It is updated by Eq. (1).
Equations. (3) and (4) are used a sigmoidal function to an
activating function which is described by Eq. (5).
f (a) =

1
1 + e−a(5)where a is an inner state.
III. Simulations

Fig. 3.

In this section, we show the experimental result. We use
seven kinds of the MLPs for comparison of the performance.
(1)
The conventional MLP(2)
The MLP with random noise(3)
The MLP with random timing pulses(4)
The MLP with same timing pulses(5)
The MLP with pulse glial chain (only propagation to
one directions)(6)
The MLP with pulse glial chain(7)
The MLP with positive and negative pulse glial
chain.
The MLP with random noise is given an uniformed random
noise to the threshold of the neurons in the hidden layer.
The MLP with random timing pulses receive the pulses at
random timing to the the threshold of the neurons. In the
MLP with random timing pulses, all glias are independent
from other glias. The MLP with same timing pulses receive
the pulse at same timing. The all glias are excited and generate
pulses at same time. These pulses are given to the thresholdof connected neurons at same time. In the MLP with pulse
glial chain (only propagation to one directions), the glias canonly propagate the pulses to one side. In the MLP with pulse
glial chain, the glias have only positive response.
We use a Mean Square Error (MSE) to the error function.
It is described by Eq. (6).

An example of glial pulses (D = 5).

1 (T n − On )2 ,N n=1
N

B. Updating rule of neuron
The neuron has multi-inputs and single output. We canchange the neuron output by the tuning the weights of connections. The standard updating rule of the neuron is definedby Eq. (3).
⎛ n
⎞
⎜⎜⎜
⎟⎟⎟
⎜
yi (t + 1) = f ⎜⎜⎝ wi j (t)x j (t) − θi (t)⎟⎟⎟⎠ ,(3)
j=1

where y is an output of the neuron, w is a weight of connection,
x is an input of the neuron, and θ is a threshold of neuron.
Next, I show a proposed updating rule of the neuron. We add
the glial eﬀect to the threshold of neuron. This updating rule isused to neurons in the hidden layer. It is described by Eq. (4).
⎛ n
⎞
⎜⎜⎜
⎟⎟⎟
(4)
yi (t + 1) = f ⎜⎜⎝⎜ wi j (t)x j (t) − θi (t) + αψi (t)⎟⎟⎠⎟ ,j=1

where α is a weight of the glial eﬀect. We can change theglial eﬀect by change of α. In this equation, the weight ofconnection and the threshold are changed by BP algorithm as

MS E =(6)

where N is a number of learning data, T is a target value,
and O is an output of MLP. We obtain results which are
an average error, a minimum error, a maximum error, anda standard deviation of the results.
A. Simulation task
In this simulation, we use the Two Spirals Problem (TSP).
The TSP is a famous benchmark of the artificial neuralnetwork [10][11]. It is known to a high nonlinearly task. Thistask has the two diﬀerent spiral points. The MLPs learn theclassification of two spiral points. We use the two diﬀerent
tasks which spirals are composed of 98 points and 130 points.
Figure 4 is two spirals for diﬀerent number of points. The
MLPs learn each coordinates of points. Moreover, we canobtain the generalization capability from solving the TSP.
We input coordinates between 0 and 1 to the after learning
MLP. We obtain the output of the network and can know thateach coordinate fits into which spirals. Figure 5 shows that2592

the ideal results of the classification of coordinates. We makethe ideal classification result by calculation of norm between
coordinates and spiral points. In this simulation, the MLP is
composed of 2-40-1. We increase the number of neurons in thehidden layer. Because, the learning performance of the MLP
is improved by the number of neurons.

(a) 98.

(b) 130.

Fig. 4.

Target points.

(a) 98.

(b) 130.

Fig. 5.

Ideal classification results.

1) Spirals composed of 98 points: First, we show theexperimental result from the learning 98 points in Table I.
The minimum error is similar value. From this result, every
MLP can search the optimum solution. Thus, the average error
shows the escaping performance from local minimum. The
average error of the MLP with negative and positive pulse glial
network is the best of all. When we compare the conventional
MLP and proposed MLP, the proposed MLP has twice thelearning performance.
TABLE I
Learning performance.
(1)
(2)
(3)
(4)
(5)
(6)
(7)

Average
0.04153
0.03711
0.03666
0.03873
0.031780.02072
0.01531

Minimum
0.000170.00006
0.000150.000220.000240.000110.00009

Maximum
0.183870.17352
0.082080.17335
0.07186
0.08192
0.06157

Std. Dev.
0.02637
0.02946
0.02195
0.02632
0.01986
0.01782
0.01636

capability. From this result, the proposed MLP has an abilityof searching global solution.
TABLE IIClassification performance.
(1)
(2)
(3)
(4)
(5)
(6)
(7)

Average
0.150290.13966
0.147020.150810.131230.12233
0.10980

Minimum
0.080850.08083
0.07965
0.07601
0.07986
0.081400.06408

Maximum
0.211270.203780.20083
0.22355
0.173760.170420.15069

Std. Dev.
0.02434
0.02879
0.02553
0.027450.021620.019390.01902

Figure 6 is examples of the classification results. We show
the near results of the average. The results of the MLP withpulse glial chain and the MLP with positive and negative pulse
glial chain can represent the two spirals. However, the others
become similar, and are cut a part of spirals. We can see thatthe boundary line of the spirals as the MLP with positive
and negative pulse glial chain is smoother than the MLP withpulse glial chain. We consider that the MLP with positive andnegative pulse glial chain has better classification ability thanthe MLP with pulse glial chain.
2) 130 spirals: In this section, we show that the result
for the MLP learning spirals of 130 points. Table III is theexperimental result of the MLPs. The results similar trend to
the 98 points. The error average of the conventional MLP
becomes large value. In the TSP, when the spirals points
increase, the task become diﬃcult. Thus, the conventional
MLP is trapped local minimum. The noise is eﬃciency forthe local minimum problem. The noise gives the energy to
the MLP, and we believe that the MLP escapes from the local
minimum. From this table, we can see that the noise pattern isimportant for the MLP learning. The noise patterns of (2), (3)and (4) is little eﬃciency for the learning performance in thissimulation. All MLPs with pulse glial chain decrease the error.
From this result, we consider that the relationship of positionof neurons is important for the MLP learning. This becauseof that the MLP with pulse glial chain is better the learning
performance than the MLP with pulse glial chain (only one
direction). Moreover, in the MLP with positive and negative
pulse glial chain, the glial eﬀects are decomposed which are
the positive part and the negative pulse part. We consider thatthe positive pulse part and the negative pulse part accentuate
the relationship of positions of neurons.
TABLE III
Learning performance.

Next, we show that the statistic classification result inTable II. In this result, the average of error is the smallest
when we use the MLP with positive and negative pulse glial
chain. Generally, the generalization capability become weak
when the MLP has over learning. The MLP with positive andnegative pulse glial chain has both the performance of function approximation and the performance of the generalization

2593

(1)
(2)
(3)
(4)
(5)
(6)
(7)

Average
0.12269
0.108470.10858
0.093910.04597
0.03830
0.01673

Minimum
0.008310.000470.004160.006020.000520.000630.00048

Maximum
0.23857
0.24278
0.268440.246160.12401
0.121900.08527

Std. Dev.
0.055540.057420.05546
0.05276
0.02537
0.025890.01661

Table IV is the statistic classification result. The MLP with

TABLE IV
Classification performance.

(a) Conventional. MLP(b) MLP with random noise(c) MLP with random timing.

(d) MLP with same timing.

(e) Proposed MLP (one direction).

(1)
(2)
(3)
(4)
(5)
(6)
(7)

Average0.21782
0.19278
0.19799
0.195640.16068
0.147310.11925

Minimum
0.10565
0.10460
0.12332
0.11603
0.086510.08792
0.06585

Maximum
0.294770.33065
0.346930.30857
0.22352
0.237230.18821

Std. Dev.
0.03858
0.044340.04086
0.03657
0.030800.02826
0.02324(a) Conventional. MLP(b) MLP with random noise(c) MLP with random timing.

(d) MLP with same timing.

(e) Proposed MLP (one direction).

(f) Proposed MLP.

(f) Proposed MLP.

(g) Proposed MLP.

Fig. 6.

Examples of classification results.

positive and negative pulse glial chain is the best of all. Thiserror average is similar to the learning 98 points.
Finally, we show the examples of classification results near
the error average in Fig. 7. The conventional MLP cannot
represent the two spirals. Figures 7 (b), (c) and (d) look liketo represent the spirals. However, they are cut some parts ofspirals. Figure 7 (e) is cut two parts of spirals. Figure 7 (f)is cut one part of spirals. We can see the two spirals in thesetwo classification results. The MLP with positive and negative
pulse glial chain can almost represent the two spiral. This
image is not cut any part. From this simulation, we consider
that the MLP with positive and negative pulse glial chain has
the highest generalization capability of all MLPs.

2594(g) Proposed MLP.

Fig. 7.

Examples of classification results.

IV. Conclusion
In this study, we have proposed the MLP with positive andnegative pulse glial chain which is inspired from the feature of
biological glia. We add the pulse glial chain to the neurons of
the hidden layer in the MLP. In the positive and the negative
pulse glial chain, the glia generates the pulse when the glia is
excited by the connected neuron. The excitation glia generates
the positive pulse when the connected neuron has the large
output. The excitation glia generates the negative pulse when
the connected neuron has the small output. We consider thatthe pulse glial chain gives the relationship of position of theconnected neurons and that this relationship improves the MLP
learning performance. By solving the TSP, we confirm that
the MLP with positive and negative pulse glial chain has thehigh learning performance and the generalization capability.
Moreover, it can clearly represent the two spirals.
Acknowledgment
This work was partly supported by JSPS Grant-in-Aid for
Scientific Research 22500203.
References[1] P.G. Haydon, “Glia: Listening and Talking to the Synapse,” Nature
Reviews Neuroscience, vol. 2, pp. 844-847, 2001.
[2] S. Koizumi, M. Tsuda, Y. Shigemoto-Nogami and K. Inoue, “Dynamic
Inhibition of Excitatory Synaptic Transmission by Astrocyte-Derived ATP
in Hippocampal Cultures,” Proc. National Academy of Science of U.S.A,
vol. 100, pp. 11023-11028, 2003.
[3] S. Ozawa, “Role of Glutamate Transporters in Excitatory Synapses inCerebellar Purkinje Cells,” Brain and Nerve, vol. 59, pp. 669-676, 2007.
[4] G. Perea and A. Araque, “Glial Calcium Signaling and Neuro-Glia
communication,” Cell Calcium, vol. 38, 375-382, 2005.
[5] S. Kriegler and S.Y. Chiu, “Calcium Signaling of Glial Cells along
Mammalian Axons,” The Journal of Neuroscience, vol. 13, 4229-4245,
1993.
[6] M.P. Mattoson and S.L. Chan, “Neuronal and Glial Calcium Signaling in
Alzheimer’s Disease,” Cell Calcium, vol. 34, 385-397, 2003.
[7] C. Ikuta, Y. Uwate and Y. Nishio, “Chaos Glial Network Connectedto Multi-Layer Perceptron for Solving Two-Spiral Problem,” Proc. ISCAS’10, pp. 1360-1363, May 2010.
[8] C. Ikuta, Y. Uwate, and Y. Nishio, “Performance and Features ofMulti-Layer Perceptron with Impulse Glial Network,” Proc. IJCNN’11,
pp.2536-2541, Jun. 2011.
[9] D.E. Rumelhart, G.E. Hinton and R.J. Williams, “Learning Representations by Back-Propagating Errors,” Nature, vol. 323-9, pp. 533-536,
1986.
[10] J.R. Alvarez-Sanchez, “Injecting knowledge into the Solution of the
Two-Spiral Problem,” Neural Computing & Applications, vol. 8, pp. 265272, 1999.
[11] H. Sasaki, T. Shiraishi and S. Morishita, “High precision learning forneural networks by dynamic modification of their network structure,”
Dynamics & Design Conference, pp. 411-1–411-6, 2004.

2595