2010 International Workshop on Nonlinear Circuits,Communication and Signal Processing
NCSP'10, Waikiki, Hawaii, March 3-5, 2010

Parameter Dependency of Chaos Glial Network
Connected to Multi-Layer Perceptron
Chihiro Ikuta† , Yoko Uwate‡ and Yoshifumi Nishio††Tokushima University
2-1 Minami-Josanjima, Tokushima, Japan
Phone:+81-88-656-7470
Email: { ikuta, nishio} @ee.tokushima-u.ac.jp‡University / ETH Zurich
Winterthurerstrasse 190, CH-8057 Zurich, Switzerland
Email: yu001@ini.phys.ethz.chAbstract
We proposed chaos glial network connected to Multi-Layer
Perceptron (MLP). The glial network is inspired by astrocyte
which is glial cell in the brain. Glial cell generates chaotic
oscillation and this oscillation is propagated other glial cells
into the glial network. We consider that this oscillation gives
good influence to MLP learning.
In this paper, we investigate chaos glial network connected
to MLP in some conditions. We apply the MLP networks forsolving Two-Spiral Problem (TSP). TSP is a problem whichclassifies two spirals drawn on the plane, and it is famousas the high nonlinear problem. By computer simulations forsolving TSP, we confirm that chaos glial network connected
to MLP has strong dependency for glial network parameters.
1. Introduction
Back Propagation (BP) was introduced by Rumelhart in1986 [1]. BP is used for learning algorithm of Multi-Layer
Perceptron (MLP) and the error propagates backwards in thenetwork. MLP using BP algorithm is well known to perform
for the pattern classification tasks. However, the solutionof the network often falls into the local minimum, because
BP uses the steepest descent method for the leaning process.
Generally, if the solution of MLP falls into the local minimum, it can not escape. In order to avoid this problem, somemethods to release the solution from the local minimum arerequired.
Recently, the mechanism of astrocyte becomes clear withthe advances in neuroscience technology. Astrocyte is a glial
cell which existing in a central nervous system of brain. Several research groups discovered that astrocytes aﬀect to neurons with signal [2]. We proposed a chaos glial network connected that to MLP and we confirmed this MLP network wasbetter performance than conventional MLP [3]. In the glial
network, glial cell connected to neighborhood glial cells andinfluence of glial cell propagates. We showed that this net-work gave good influence to MLP learning.
In this study, we investigate chaos glial network connected
to MLP in some conditions. We clearly discuss parametricdependency of chaos glial network connected to MLP. We
apply the MLP to the TSP [4] and confirm the eﬃciency bycomputer simulations.

2. Multi-Layer Perceptron
MLP is the most famous feed forward neural network. Thisnetwork is used for pattern recognition, pattern classification,and other tasks. MLP has some layers, it has mainly inputlayer, hidden layer, and output layer. Generally, it is known
that MLP can solve a more diﬃcult task if the number of layeror neuron is increased. We consider MLP which is composedof four layers (one input layer, two hidden layers and one output layer), and MLP has the glial network in the second layerof the hidden layer. The proposed glial network connected toMLP structure (connected 2-20-40-1) is shown in Fig. 1.

- 596 -Figure 1: Chaos glial network connected to MLP.

2.1. Neuron Updating Rule
The updating rule of neurons in the input layer, the first hidden layer and the output layer is described by Eq. (1) whichis conventional updating rule.
 n
∑


xi (t + 1) = f  wi j (t)x j (t) − θi (t)
(1)
j=1

In the chaos glial network connected to MLP, chaotic oscillation is added to neurons in the second hidden layer. This
neuron’s updating rule is following as Eq. (2).
 n
∑


xi (t + 1) = f  wi j (t)x j (t) − θi (t) + αΨi (t) , (2)j=1where x : input or output, w : weight parameter, θ : threshold,ψ, Ψ : random oscillation, α : amplitude of random noiseand f : output function. And we use sigmoid for the output
function as Eq. (3).
f (a) =1
1 + e−a(3)

In the biological neural network, it is known that the glial
cells aﬀect to the neighbor neurons over a wide range by propagating in the network [5]. In order to realize phenomena, weadd chaotic oscillation to neurons by using Eq. (4). In thissimulation, we use skew tent map which is given Eq. (5).
m∑Ψi (t) =β|k| ψi+k (t − k),Figure 2: Propagating chaotic oscillation in the network.
where E: error value, t: target value and O: neuron output. Bychanging the value of weights, MLP’s error becomes smaller.
Therefore, partial diﬀerential of the weight is carried out byEq. (7).
∆Wk j = −η

(4)

k=−m

 2ψ(t)+1−A (−1 ≤ ψ(t) ≤ A)
1+A
ψi (t + 1) = , −2ψ(t)+1+A(A < ψ(t) ≤ 1)
1−A

∂E
∂wk j(7)3. Simulations(5)

where β denotes attenuation parameter and k is the propagating range in the glial network. Chaotic oscillation is propagating in the glial network as shown in Fig. 2 and it takes timedepending on distance. For example, when the glial cell is located three units far from certain neuron, eﬀect of chaotic oscillation arrives to the neuron after three learning steps. And
chaotic oscillation decreases while chaos is propagating in thenetwork.

In this section, the diﬀerence in the performance of ourMLP; We investigate behavior of MLP as changing glial network’s parameters.
3.1. Two-Spiral Problem
We apply the proposed network for solving TSP [4]. MLPlearns to each point of two spirals, and MLP learns by using the standard BP algorithm. In this simulation, we use 98
points of two spiral for learning.
3.2. Simulation Results

2.2. Back Propagation
The error of MLP propagates backward in the feed forward
neural network. BP algorithm changes value of weights toobtain smaller error than before. The error of the network is
given by Eq. (6).
1∑(ti − Oi )2 ,2 i=1Each MLP learns the two spirals by setting up same
weights before learning process. We prepare 98 data of twospirals as shown in Fig. 3. The number of learning points isfixed as 500000. We investigate the error which is modified
in the meantime. The error function is defined as Eq. (8).

nE=1∑|ti − Oi |
n i=1
n(6)

- 597 -E=(8)Figure 3: Two-spiral problem.
Figure 4: Error curve by MLP network as changing m.
3.2.1. m Dependency
We prepare 98 data of two spirals. The number of learning
points is fixed as 500000. We change the propagating range m(5, 10, 20 and 40) in the Eq. (4) and we use β = 0.8. Table 1
shows simulation result of each MLP learn to 98 points.

ble 2 shows result of each MLP as changing β (0.2, 0.4, 0.6,0.8 and 1.0).
Table 2: Performance of MLP with chaos glial network bychanging attenuation parameter.

Table 1: Performance of MLP with chaos glial network bychanging propagating range of chaotic oscillations.
Points5102040

100000200000250000300000350000400000450000500000

0.318
0.194
0.152
0.103
0.092
0.078
0.075
0.068

0.318
0.230
0.182
0.129
0.108
0.085
0.075
0.077

0.299
0.197
0.149
0.084
0.065
0.050
0.043
0.035

0.299
0.200
0.153
0.097
0.073
0.069
0.044
0.036

Points

0.2

0.4

0.6

0.8

1.0

100000200000250000300000350000400000450000500000

0.309
0.220
0.164
0.133
0.104
0.077
0.064
0.060

0.291
0.198
0.136
0.114
0.087
0.078
0.072
0.062

0.307
0.206
0.160
0.112
0.0900.073
0.066
0.053

0.299
0.197
0.149
0.084
0.065
0.050
0.043
0.035

0.396
0.301
0.258
0.229
0.231
0.195
0.188
0.179

We can see that MLP obtains the best result when the attenuation parameter is set to β = 0.8. While, in the case of
β = 1.0, MLP shows the worst result. Moreover, the results
From this table, when the propagating range is fixed as become better by increasing the value of β expect β = 1.0.
m=20, 40, MLP gains better performance than the case of We consider that influences of other glial cells are important
m=5, 10. We consider that propagating oscillation is becom- for MLP learning.
ing small by β in the far glial cell.
Figure 5 is typical result of changing the attenuation paFigure 4 is typical result as using each m. In this result, rameter. The learning curve of β = 1.0 is very vibrating beMLP learning curve converged earlier as propagating oscilla- cause chaotic oscillation is too large.
tions are more wide range.
3.2.2. β Dependency

3.2.3. η Dependency

We showed this MLP network gives the best result to use
In this simulation, we use m = 20 from before simulation m = 20 and β = 0.8. In this section, we investigate learning
result. The number of learning points is fixed as 500000. Ta- coeﬃcient dependency of our MLP. We use diﬀerent learning- 598 -

Figure 5: Error curve by MLP networks as changing β.

Figure 6: Error curve by MLP networks as changing η.

coeﬃcient (η = 0.025, 0.05, 0.1, 0.2, 0.4). From this table,chaotic oscillations to the second hidden layer’s neuron. We
confirmed that chaos glial network connected to MLP had
strong dependency for glial network parameter.

Table 3: Performance of MLP with chaos glial network bychanging learning coeﬃcient.
Points

0.025

0.05

0.10

0.20

0.40

100000200000250000300000350000400000450000500000

0.396
0.351
0.342
0.329
0.321
0.314
0.309
0.303

0.299
0.197
0.149
0.084
0.065
0.050
0.043
0.035

0.158
0.071
0.051
0.048
0.037
0.064
0.082
0.089

0.146
0.148
0.222
0.198
0.215
0.234
0.234
0.233

0.477
0.498
0.489
0.495
0.5080.492
0.495
0.494

References[1] D.E. Rumelhart, G.E. Hinton and R.J. Williams, “Learning Representations by Back-Propagating Errors,” Nature, vol.323-9, pp.533-536, 1986.
[2] P.G. Haydon, “Glia: Listening and Talking to theSynapse,” Nature Reviews Neuroscience, vol. 2, pp.844847, 2001.
[3] C. Ikuta, Y. Uwate and Y. Nishio, “Chaos Glial Network
Connected to Multi-Layer Perceptron for Solving TwoSpiral Problem,” Proc. ISCAS’10, May 2010. (Accepted)[4] J.R. Alvarez-Sanchez, “Injecting knowledge into the Solution of the Two-Spiral Problem,” Neural Computing &we confirm that the error of MLP can converge fast whenApplications, vol.8, pp.265-272, 1999.
the learning coeﬃcient is set to the large value. However, if
the parameter of the learning coeﬃcient is too large such as [5] S. Koizumi, M. Tsuda, Y. Shigemoto-Nogami and K. Inη = 0.2, 0.4, MLP hardly learn to the two-spiral. When theoue, “Dynamic Inhibition of Excitatory Synaptic Translearning coeﬃcient is very small (η = 0.025), MLP could notmission by Astrocyte-Derived ATP in Hippocampal Culfinish learning at 500000 times.
tures,” Proc. National Academy of Science of the U. S. A,
Figure 6 shows one example of the error curve as changing
vol.100, pp.11028-11028, 2003.
η. When the learning coeﬃcient is fixed as η = 0.4, the error
curve vibrates and MLP could not learn the two-spiral. In thecase of η = 0.05, 0.1, 0.2, the learning curves are very smooth
and when learning coeﬃcient is large, convergence of error isfast.
4. Conclusions
In our study, we have investigated parameter dependencyof chaos glial network connected to MLP. This network gave

- 599 -