IEEE Workshop on Nonlinear Circuit Networks
December 9-10, 2011

Multi-Layer Perceptron with Pulse Glial Chain for
Solving Two-Spirals Problem
Chihiro Ikuta

Yoko Uwate

Yoshifumi Nishio

Dept. of Electrical and Electronics Eng., Dept. of Electrical and Electronics Eng., Dept. of Electrical and Electronics Eng.,
Tokushima University
Tokushima University
Tokushima University
2-1 Minami-Josanjima,
2-1 Minami-Josanjima,
2-1 Minami-Josanjima,
Tokushima Japan
Tokushima Japan
Tokushima Japan
Email: ikuta@ee.tokushima-u.ac.jp
Email: uwate@ee.tokushima-u.ac.jp
Email: nishio@ee.tokushima-u.ac.jp

I. I NTRODUCTIONII. P ROPOSED M ETHOD
The MLP is the most famous feed forward neural network.
This network is composed of layers of neurons and its outputs
are controlled by the weights of connections. In general itis learned by Back Propagation algorithm (BP) which wasproposed by D.E. Rumelhart [8]. We connect the glias to theneurons in hidden layer. We show the proposed MLP in Fig. 1.
A. Glial Pulse
The neuron is investigated by many researchers and its
applications are proposed. However, the glia is not investigated- 51 -Neuron…

Currently, a glia which is one of nervous cell is known tohave important works in a brain [1]. The glias has several ion
channels, for example calcium ion Ca2+ , adenosine triphosphate (ATP), glutamine acid, and so on [2]-[4]. These ionsare also used by neurons in gap junctions. Moreover, Ca2+
affect around glias’ states and neurons’ thresholds [5][6]. We
consider that the glia has relationships with brain works. Fromthese futures, we have noticed the glia works and have tried
applications to artificial neural networks.
In the previous study, we proposed a Multi-Layer Perceptron (MLP) with pulse glial chain from the features of thebiological glia [7]. In that model, the glias connected neurons
one-by-one in a hidden layer of the MLP. The glias are excitedby huge amount of output of connected neuron. The excitation
glia generates the pulse output, moreover the pulse excite theneighborhoods glias and affects the threshold of the connected
neuron. We believed that the glias give the relationships ofpositions of neurons in hidden layer of neurons and thatthe glias improve the learning performance of the MLP. We
confirmed that the MLP with pulse glial chain had better
learning performance than the conventional MLP by learningof successive chaotic time series.
In this study, we apply the MLP with pulse glial chain to
Two-Spirals Problem (TSP). The TSP is a famous benchmark
for artificial neural networks. This task is known to have ahigh nonlinearity. We confirm that the MLP with pulse glialchain has high solving ability as nonlinear problem.

GliaFig. 1.

MLP with pulse glial chain.

details, because this cell was know to static cell in thebrain. Currently, the glia attracts researchers attentions in thebiological or the medical fields. Because, the glia actually has
several important works in the brain. The glia has several ionschannels. Especially, the glia can change the Ca2+ density andthis Ca2+ density become pulse response [5][6]. The Ca2+change the threshold of the neuron. Moreover, this influence
propagates wide range in the brain.
We have proposed the pulse glial chain which is inspired
from these features of biological glia. All glias connect theneurons in the hidden layer and generate pulses by the outputsof connected neurons. We define an output function of the glia
in Eq. (1).
ψi (t + 1) ={ 1, {(θ < y ∪ ψnii+1,i−1 (t − i ∗ D) = 1)

∩ (θg > ψi (t))} ,(1)γψi (t), else,
where ψ is an output of a glia, γ is an attenuated parameter,
y is an output of a connecting neuron, θn is a glia thresholdof excitation, θg is a period of inactivity, and D is a delay
time of a glial effect. The glias do not learn by BP algorithm,
however, the neurons are learned by BP algorithm. Thereby,
the generation pattern of glial pulse is dynamically changeduring MLP learning.

B. Simulation results

B. Updating Rule of Neuron

In this simulation, we use three layers MLPs which arecomposed of 2-40-1. The MLPs learn the 50000 times during
the one trial. We obtain results from 100 trials. We use a
Mean Square Error (MSE) to the major of performance. It is
described by Eq. (5).

The neuron has multi-inputs and single output. We can process the neuron output by the changing weights of connections.
The standard updating rule of the neuron is defined by Eq. (2).
n∑yi (t + 1) = f wij (t)xj (t) − θi (t) ,(2)1 ∑M SE =(Tn − On )2 ,N n=1
N

j=1

where y is an output of the neuron, w is a weight ofconnection, x is an input of the neuron, and θ is a thresholdof neuron. In this equation, the weight of connection andthreshold of neuron are learned by BP algorithm. Next, Ishow a proposed updating rule of the neuron. We add theglial effect to the threshold of neuron. This updating rule isused to neurons in hidden layer. It is described by Eq. (3).
n∑yi (t + 1) = f wij (t)xj (t) − θi (t) + αψi (t) ,(3)j=1where α is weight of glial effect. We can control the glial effectby change of α. In this equation, the weight of connection
and the threshold are learned by BP algorithm as same as thestandard updating rule of the neuron. However, the glial effectdoes not learn. It is updated by Eq. (1).
Eqs. (2) and (3) are used sigmoidal function to activating
function which is described by Eq. (4).
f (a)

=1
1 + e−a(4)where a is an inner state.
III. S IMULATIONIn this section, we confirm the learning performance of theproposed MLP and generalization ability. We use five kindsof MLPs for a comparison of the performance as follows.
(1)
The conventional MLP(2)
The MLP adding random timing pulses(3)
The MLP adding same timing pulses(4)
The MLP with pulse glial chain (one direction)(5)
The MLP with pulse glial chain
In the MLP adding random timing pulses, this MLP is given
the pulses to the thresholds of neurons at random timing.
The random timing pulses are decorrelation each other. TheMLP adding same timing pulses is given the pulses which aregenerated at same timing.

where N is a number of learning data, T is a target value, andO is an output of MLP.
We show the learning performance of the MLPs fromlearning 98 points in Table I. The learning performance of
MLP with pulse glial chain is the best of all as the average oferror. The learning performance of the conventional MLP isthe worst, this because of the TSP has the high nonlinearity,
thus this task has many local minima. We consider that theconventional MLP is trapped the local minimum. The MLP
with pulse glial chain (only one direction) becomes second
winner from the comparison of the averages. From theseresults, we can say that the pulse glial chain is more effective
than the pulse glial chain (one direction). We consider thatthe pulse glial chain give more relationships of position ofneurons to the MLP.
Table II is the classification results. It means the generalization ability of the MLPs. This table shows that theperformance of the generalization ability is similar to thelearning performance. In general, when the MLP becomesover learning, its generalization ability is decreased. However,in this result, the generalization ability of the MLP pulse
glial chain is the best of all. Thereby, the MLP with pulse
glial chain does not become over learning. An example of theclassification results as around average are shown in Fig. 2.
TABLE IL EARNING PERFORMANCE .
(1)
(2)
(3)
(4)
(5)

- 52 -Average
0.04153
0.03666
0.03873
0.031780.02072

Minimum
0.000170.000150.000220.000240.00011

Maximum
0.18387
0.082080.17335
0.07186
0.08192

Std. Dev.
0.02637
0.02195
0.02632
0.01986
0.01782

TABLE IIC LASSIFICATION PERFORMANCE .

A. Simulation Task
We use the TSP for simulation task. It is a famous benchmark of the neural networks and is known to high nonlinearity
problem. In this task, the MLP learns the classifications of two
different spirals which are composed of many points. The MLP
is inputted the coordinates of spiral points, and is learned byideal classifications.

(5)

(1)
(2)
(3)
(4)
(5)

Average
0.150290.147020.150810.131230.12233

Minimum
0.080850.07965
0.07601
0.07986
0.08140

Maximum
0.211270.20083
0.22355
0.170420.17042

Std. Dev.
0.02434
0.02553
0.027450.021620.01939

R EFERENCES(a) Conventional. MLP(b) MLP adding random timing.

(c) MLP adding same timing.

(d) Proposed MLP (one direction).

(e) Proposed MLP.

Fig. 2.

Examples of classification results.

IV. C ONCLUSIONSIn this study, we have investigated the learning performanceof the MLP with pulse glial chain using by the TSP. The
pulse glial chain is inspired from features of biological glia.
All glias generate the pulse output when the glias are excitedby output of connecting neurons. The pulses are propagatedby glia, after that, the pulses influence to the threshold ofneurons. We connected the glias to the neurons in hidden layerof MLP. We confirmed that the MLP with pulse glial chain had
better learning performance than the conventional MLP. Fromthe simulations, we showed that the MLP obtained the highlearning performance by to be given relationships of positionof neurons. Moreover, we investigated generalization ability of
the MLPs by classification of two spirals. We show that theMLP with pulse glial chain has the high generalization ability
and can represent the spirals. From this study, we confirmedthat the MLP with pulse glial chain was suitable for use thehigh nonlinearity tasks.

- 53 -[1] P.G. Haydon, “Glia: Listening and Talking to the Synapse,” NatureReviews Neuroscience, vol. 2, pp. 844-847, 2001.
[2] S. Koizumi, M. Tsuda, Y. Shigemoto-Nogami and K. Inoue, “DynamicInhibition of Excitatory Synaptic Transmission by Astrocyte-Derived
ATP in Hippocampal Cultures,” Proc. National Academy of Science ofU.S.A, vol. 100, pp. 11023-11028,[3] S. Ozawa, “Role of Glutamate Transporters in Excitatory Synapses inCerebellar Purkinje Cells,” Brain and Nerve, vol. 59, pp. 669-676, 2007.
[4] G. Perea and A. Araque, “Glial Calcium Signaling and Neuro-Glia
Communication,” Cell Calcium, vol. 38, 375-382, 2005.
[5] S. Kriegler and S.Y. Chiu, “Calcium Signaling of Glial Cells along
Mammalian Axons,” The Journal of Neuroscience, vol. 13, 4229-4245,
1993.
[6] M.P. Mattoson and S.L. Chan, “Neuronal and Glial Calcium Signalingin Alzheimer’s Disease,” Cell Calcium, vol. 34, 385-397, 2003.
[7] C. Ikuta, Y. Uwate and Y. Nishio, “Multi-Layer Perceptron with Pulse
Glial Chain,” Proc. NOLTA’11, pp. 435-438, Sep. 2011. 2003.
[8] D.E. Rumelhart, G.E. Hinton and R.J. Williams, “Learning Representations by Back-Propagating Errors,” Nature, vol. 323-9, pp. 533-536,
1986.