Chaos Glial Network Connected toMulti-Layer Perceptronfor Solving Two-Spiral Problem
Chihiro Ikuta

Yoko Uwate

Yoshifumi Nishio

Dept. of Electrical and Electronic Eng.,
Tokushima University
2-1 Minami-Josanjima, Tokushima, Japan
E-mail: ikuta@ee.tokushima-u.ac.jp

University / ETH Zurich,
Winterthurerstrasse 190,
CH-8057 Zurich, Switzerland,
Email: yu001@ini.phys.ethz.ch

Dept. of Electrical and Electronic Eng.,
Tokushima University
2-1 Minami-Josanjima, Tokushima, Japan
E-mail: nishio@ee.tokushima-u.ac.jp

Abstract— Some methods using artificial neural network wereproposed for solving to the Two-Spiral Problem (TSP). TSP is aproblem which classifies two spirals drawn on the plane, and it
is famous as the high nonlinear problem.
In this paper, we propose a chaos glial network whichconnected to Multi-Layer Perceptron (MLP). The chaos glial
network is inspired by astrocyte which is glial cell in the brain.
By computer simulations for solving TSP, we confirmed that theproposed chaos glial network connected to MLP gains better
performance than the conventional MLP.

I. I NTRODUCTION
Back Propagation (BP) was introduced by Rumelhart in1986 [1]. BP is used for learning algorithm of MLP and theerror propagates backwards in the network. MLP using BP algorithm is well known to perform for the pattern classification
tasks. However, the solution of the network often falls into the
local minimum, because BP uses the steepest descent method
for the leaning process. Generally, if the solution of MLP falls
into the local minimum, it can not escape. In order to avoid
this problem, some methods to release the solution from thelocal minimum are required.
Recently, the mechanism of astrocyte which is glial cell
existing in the central nervous system of the brain has been
attracting. Several research groups discovered that astrocytes
affect to neurons with signal transduction [2]. We consider
that astrocytes make good effects to neurons in the biologicalneural networks.
In this study, we propose a chaos glial network whichconnects to MLP as shown in Fig. 1. We consider that glial
cells produce chaotic oscillation which is affected to neurons.
This view is motivated by investigations of the Hopfield
network solving combinatorial optimization problems with thehelp of a chaotic input signal component, designed in order
to avoid local minima. It appears, from computer simulations,that a chaotic input component may substantially enhance thecapability of avoiding these local minima [3]-[5]. Hence, webelieve that chaotic signals may be used to further enhancethe efficiency of the proposed chaos glia neural network.
Furthermore, chaotic oscillation generated from glial cells
propagates to the neighbor glial cells. Namely, certain neuron

978-1-4244-5309-2/10/$26.00 ©2010 IEEEin this network is affected from some of glial cells locatedat a nearby site. We apply the proposed chaos glial network
connected to MLP for solving TSP and confirm the efficiency
by computer simulations.
Glial Cell
GlialNetworkChaos
MLP
NeuronFig. 1.

Conceptual chaos glial network.

II. M ULTI -L AYER P ERCEPTRON
MLP is a most famous feed forward neural network. Thisnetwork is used for pattern recognition, pattern classification,and other tasks. MLP has some layers, it has mainly input
layer, hidden layer, and output layer. Generally, it is known
that MLP can solve a more difficult task if the number of layeror neuron is increased. We consider MLP which is composedof four layers (one input layer, two hidden layers and oneoutput layer), and MLP has the chaos glial network in thesecond layer of the hidden layer. The proposed MLP with thechaos glial network structure (connected 2-20-40-1) is shownin Fig. 2.
A. Neuron Updating Rule
The updating rule of neurons in the input layer, the first
hidden layer and the output layer is described by Eq. (1) whichis conventional updating rule.
⎛⎞n
wij (t)xj (t) − θi (t)⎠ ,(1)xi (t + 1) = f ⎝j=1

In the chaos glial neural network, chaotic oscillation isadded to neurons in the second hidden layer. This neuron’s

1360

t=1

Neural network

Glial Cell

Propagating
OscillationGlia
al network

xNeuront=2・・・・・・

yO

t=3

Input Layer

Fig. 2.

Hidden Layer

Output Layer

Fig. 3.

Chaos glial network connected to MLP.

updating rule is following as Eq. (2).
⎛⎞nxi (t + 1) = f ⎝wij (t)xj (t) − θi (t) + αΨi (t)⎠ ,(2)Therefore, partial differential of the weight is carried out by
Eq. (6).
ΔWkj = −η

j=1where x : input or output, w : weight parameter, θ : threshold,ψ, Ψ : chaotic oscillation, α : amplitude of chaos and f :output function. And we use sigmoid for the output function
as Eq. (3).
1(3)
1 + e−aIn the biological neural network, it is known that theglial cells affect to the neighbor neurons over a wide range
by propagating in the network [7]. In order to realize thisphenomena, we add chaotic oscillation to neurons by using
Eq. (4).
f (a)Ψi (t)==m

β |k| ψi+k (t − |k|),Propagating chaos in the network.

∂E∂wkj(6)

III. C HAOTIC O SCILLATION OF G LIAL N ETWORK
We use skew tent map to generate chaos. This map is
one of simple chaotic maps and the center of map is shifted
for a little from the standard tent map. This chaotic map isdefined by Eq. (7) and the diagram of this map is shown inFig. 4. Neurons in the second hidden layer are affected chaotic
oscillation by Eq. (7) into the Eq. (4).
ψi (t + 1) =

 2ψ(t)+1−A (−1 ≤ ψ(t) ≤ A)
1+A−2ψ(t)+1+A
1−A,

(7)

(A < ψ(t) ≤ 1)

Here, we consider the case that we prepare two similar initial
(4)

k=−m

where β denotes attenuation parameter and k is the propagating range in the glial network. Chaotic oscillation ispropagating in the glial network as shown in Fig. 3 and it takes
time depending on distance. For example, when the glial cell
is located three units far from certain neuron, effect of chaoticoscillation arrives to the neuron after three learning steps. And
chaotic oscillation decreases while chaos is propagating in thenetwork.
B. Back Propagation
The error of MLP propagates backward in the feed forward
neural network. BP algorithm changes value of weights toobtain smaller error than before. The error of the network isgiven by Eq. (5).
E=n1

2 i=1

(ti − Oi )2 ,(5)where E: error value, t: target value and O: neuron output. Bychanging the value of weights, MLP’s error becomes smaller.

Fig. 4.

Skew tent map (A=0.05).

values of chaos map for adding two adjacent neurons. Figure 5
shows the two time series obtained from the skew tent map
when initial values are fixed as ψ2 (0) = 0.12459 and ψ2 (0) =0.12460. From this figure, the plots of the two time series
are spread and difference between two time series becomeslarge with time, even if we use similar initial value to generate1361

Fig. 5.

Fig. 7.

Time series of skew tent map.

chaos. This is typical phenomena of chaos known as butterfly
effect. If glial cells add these chaos to neurons, each neuron
is affected with a completely different oscillation.
However, in the real biological neuro-glial network, glialcells affect each other with neighbor cells. We use the chaos
glia propagating equation (Eq. 4) for having correlation eachother of neighbor glial cells. Figure 6 shows two chaotic timeseries by using Eq. (4). The parameters are fixed as β = 0.8,m = 5. Two chaotic time series have similar peek points. We
consider that these chaotic time series have some correlations.

Two-spiral problem.

B. Simulation Result
Each MLP learns the two spirals by setting up same weights
before learning process. We prepare 98 data of two spirals
as shown in Fig. 7. The number of learning points is fixed
as 500000. We investigate the error which is modified in themeantime. The error function is defined as Eq. (8).
nE=1|ti − Oi |,n i=1(8)where E : error value t : target value, O : output value.
C. Changing the number of neuron adding chaos
We investigate the learning ability when the number ofneuron adding chaos is changed. In this simulation, we addchaos to the neurons in the second hidden layer’s. We change
the percentage of neuron adding chaos as follows 0%, 20%,50%, and 100%. The average result for 30 times of simulations
is summarized in Tab. I.
TABLE IT HE P ERFORMANCE OF MLP WITH C HAOS G LIAL N ETWORK BY
C HANGING THE N UMBER OF N EURON A DDING C HAOS

Fig. 6.

Time series of skew tent map with correlation.

IV. S IMULATIONSIn this section, the difference in the performance of ourMLP; chaos glial network connected to MLP and the conventional MLP is compared.
A. Two-Spiral Problem
We apply the proposed network for solving TSP [6]. MLP
learns to each point of two spirals, and MLP learns by using
BP algorithm.

Points

0%

20%

50%

100%

1000001500002000002500003000003500004000004500005000000.306195
0.228353
0.170263
0.135390
0.120928
0.100454
0.090179
0.088718
0.094303

0.334227
0.269349
0.198626
0.153809
0.140445
0.136136
0.120175
0.121040
0.1091100.334600
0.224780
0.177127
0.147403
0.116600
0.111645
0.111659
0.094112
0.086106

0.322175
0.244479
0.166893
0.143317
0.115145
0.103821
0.081798
0.086527
0.074204

Minimum

0.018176

0.010911

0.002383

0.002523

From Tab. I, we obtain the best learning ability when all neurons in the second hidden layer are added chaotic oscillation.
In the next simulation, we add chaotic oscillation to every
neurons in the second hidden layer.

1362

D. Results of Each MLP LearningE. Classification of Two Spirals

We compare the chaos glial network connected to MLP, theconventional MLP and the random glial network connected
to MLP. In the random glial network, glial network produces
random oscillation. Each result is summarized in Tab. II. This
result shows the average result for 50 times of simulations.

In this section, the chaos glial network connected to MLP
classifies the two spirals. The network learns to two spiralpoints by propagating chaos method. After learning, we puts
points (0.2 ≤ x ≥ 0.8, 0.2 ≤ y ≥ 0.8) changing little by littlein MLP’s input layer. Figure 9 show the simulation result. In
this case, the error value is around 0.05 at 500000 learningpoints. When MLP’s output is nearer 1, this point is classified
to red spiral. And when MLP’s output is nearer 0, this point isclassified to blue spiral. This result shows that the chaos glial
network connected to MLP can solve TSP.

TABLE II
T HE P ERFORMANCE BY U SING E ACH MLP
PointConventional

Random Noise

Chaotic Noise

1000001500002000002500003000003500004000004500005000000.274507
0.1892180.142849
0.111989
0.084422
0.093832
0.081747
0.079630
0.076502

0.299191
0.228571
0.165421
0.136988
0.118154
0.098092
0.086327
0.087916
0.083399

0.299990
0.247936
0.197222
0.152111
0.120912
0.097169
0.080341
0.074895
0.067850

Minimum

0.0125310

0.0104630

0.0072510

At start learning, conventional MLP is better than the others.
However, after 400000 learning points the chaos glial network
connected to MLP gains better performance than the conventional MLP. The random glial network connected to MLPshows the most worst result. If we focus on the minimum error,
the chaos glial network connected to MLP can find smallest
error value. Because similarity of near neurons performs wellby effects of chaos glial cells. And when MLP falls into local
minima, chaos helps to escape with effectively.
Figure 8 is a typical example of learning curve. The
random glial network connected to MLP can not learn to twospirals. The chaos glial network connected to MLP and theconventional MLP can learn to it, and the chaos glial network
converges to the lower error value. The learning curve of the0.5
Chaos glial network

0.4

Conventional

0.35

Random glial network

Error

0.3
0.25

0.15

Conventional

0.1

RandomGlial network

0.05
00

100000200000300000400000The number of learning points

Fig. 8.

The error curve by three MLP networks.

Classification of two spirals.

V. C ONCLUSION
In our study, we have proposed a chaos glial network. Thisnetwork gave chaotic oscillations to the second hidden layer’s
neuron and this chaotic oscillation propagates to other neurons.
We confirmed that the chaos glial network connected to MLP
gains better performance than the conventional MLP and into
the random glial network connected to MLP for solving TSP.
R EFERENCES

0.45

0.2

Fig. 9.

500000conventional MLP is smooth. While, the others have littlevibration. We consider that this vibration phenomena makes
to escape out from the local minimum.

[1] D.E. Rumelhart, G.E. Hinton, and R.J. Williams, “Learning representations by back-propagating errors,” Nature, vol. 323-9, pp. 533-536, 1986.
[2] P.G. Haydon, “Glia: listening and talking to the synapse,” Nature Reviews
Neuroscience, vol 2, pp. 844-847, 2001.
[3] Y. Hayakawa, A. Marumoto, and Y. Sawada, “Effects of the chaotic noiseon the performance of a neural network model for optimization problems,”
Physical Reviews E, vol. 51, no. 4, pp. 2693-2696, Apr. 1995.
[4] T. Ueta, Y. Nishio, and T. Kawabe, “Comparison between Chaotic Noiseand Burst Noise on Solving Ability of Hopfield Neural Networks,” Proc.
NOLTA’97, vol. 1, pp. 409-412, Nov. 1997.
[5] Y. Uwate, Y. Nishio, T. Ueta, T. Kawabe, and T. Ikeguchi, “Performanceof Chaos and Burst Noise Injected to the Hopfield NN for Quadratic
Assignment Problems,” IEICE Transactions on Fundamentals, vol. E87A, no. 4, pp. 937-943, Apr. 2004. 2004.
[6] J.R. Alvarez-Sanchez, “Injecting knowledge into the solution of the twospiral problem,” Neural Computing & Applications, vol. 8, pp. 265-272,
1999.
[7] S. Koizumi, M. Tsuda, Y. Shigemoto-Nogami, and K. Inoue, “Dynamicinhibition of excitatory synaptic transmission by astrocyte-derived ATPin hippocampal cultures,” Proc. National Academy Science of the U. S.
A, vol. 100, pp. 11028-11028, 2003.

1363