平成21年度電気関係学会四国支部連合大会

333

18-9

Back Propagation with Chaotic Changeof Thresholds for Two-Spiral Problem
Chihiro IKUTA†
Yoko UWATE‡
Yoshifumi NISHIO††‡(Tokushima University) (UNI/ETH Zurich)

1. Introduction

3. Simulation Results

Back Propagation (BP) was introduced by Rumelhart
in 1986 [1]. BP is used for learning of Multi-Layer Perceptron (MLP) and the error is propagating backward in thenetwork. MLP after BP learning is known to be possible toperform pattern classiﬁcation tasks. However, network’soutput often falls into local minimum, because BP uses
the steepest descent method. We need to adopt a method
to release the solution from the local minimum.
In this study, we propose a method using independent
chaotic oscillation for the neuron units’ thresholds. The
conventional MLP’s thresholds are decided by BP learning. In our method, the thresholds are updated by BP
and also aﬀected by independent chaotic oscillations. We
apply the proposed method to the Two-Spiral Problem
and conﬁrm the eﬃciency by computer simulations.

We apply the proposed method to the two-spiral problem. This problem is famous for its diﬃculty [2]. We use
98 data of two spirals and they are shown in Fig. 2. The
maximum learning time of MLP is 50000 and we checked
the averaged error of the output in every 50 learnings.

Figure 2: Data of two-spiral problem.

2. Updating Rule of Neuron
We consider the MLP composed of four layers (connected 2-20-40-1, Fig. 1). The updating rule of the network is given by Eq. (1). The sigmoidal function (2) isused for the output function.
xi (t + 1)=f( n∑)wij (t)xj (t) − θi (t) − αζi (t) (1)

j=1

f (a)

=1
1 + e−aFigure 3 shows the learning curves for the conventional
BP learning and the proposed method with chaotic changeof the thresholds. We can see that the conventional BP
learning is trapped into a local minimum and that theperformance cannot be improved. However, the proposed
method makes the network to escape out from the local
minimum and obtain better performance.

(2)where x: input or output, w: weight parameter, θ: threshold, ζ: chaotic oscillation, α: amplitude of chaos. Squareerror (3) is used for error evaluation in BP.
1∑(ti − Oi )22nE=(3)i=1

t: target value, O: output value. We use the tent map (4)to produce chaotic oscillation and the values are exploited
to ﬂuctuate the threshold value of each unit.
ζi (t + 1) ={ 2ζ (t)i

2 − 2ζi (t)(0 < ζi (t) ≤ 1/2)
(1/2 < ζi (t) < 1)(4)

We chose diﬀerent initial values for ζi , so diﬀerent chaotic
sequences aﬀect every neurons.

Figure 1: Network composition.

Figure 3: Simulation result.

4. Conclusions
In this study, we have proposed a method to adopt
chaotic oscillations to neuron units’ thresholds. Computer
simulation results of two-spiral problem showed that theproposed method had an ability for the network to escapeout from the local minimum and obtained better results.
We consider that the drop of the threshold might be related with the existence of the Glia cells. The detailed
investigation of the eﬀect is our future work.
References[1] D.E. Rumelhart, G.E. Hinton, and R.J. Williams,“Learning representations by back-propagating errors,”
Nature, vol.323-9, pp.533-536, 1986.
[2] J.R. Alvarez-Sanchez, “Injecting knowledge into thesolution of the two-spiral problem,” Neural Computing &Applications, vol.8, pp.265-272, 1999.