平成20年度電気関係学会四国支部連合大会

383

17-14
Chaotic Backpropagation Learning Algorithm
in Feedforward Neural Network
Azian Azamimi† , Yoko Uwate‡ and Yoshifumi Nishio†(† Tokushima University, Japan and ‡ ETH Zurich, Switzerland)

1. Introduction
The backpropagation (BP) learning algorithm has
proved to be eﬃcient in many engineering applications
especially in pattern recognition and system control.
Although the BP has been a signiﬁcant research area of
artiﬁcial neural network, this algorithm has a weakness
for performing a poor convergence rate.
In this study, we propose a new modiﬁed BP learning algorithm by adding chaotic noise into weight update process. By computer simulations, we conﬁrm
that our proposed algorithm can give a better convergence rate in learning performance than the conventional BP algorithm.

2. Chaotic BP Algorithm
Slow convergence rate is the most common disadvantage of the BP learning algorithm. In the BP algorithm, the weights updates are proportional to the error propagating from output through the sigmoid function and through the weights. The convergence rate of
the learning process can be improved by changing how
the error propagates back through the network. Considering this, we add chaotic noise into weight updateprocess during error propagation. The new modiﬁed
weight update process for the BP algorithm can be
described as follows where β is the noise amplitude.
k−1,k4p wi,j(m) = −η(a) Network structure(b) Learning example

Figure 1: Network structure and learning example.

network-3 (both of them). We also compare the learning performance with the conventional method where
there is no chaotic noise application at all. Figure 2
show the learning performance of all proposed and conventional network respectively when the noise amplitude β is ﬁxed as 0.01. Here, we make two comparisons
between fully developed chaos noise and intermittencychaos noise. The logistic map is used to generate thechaotic noise where α = 4.00 shows the fully-developed
chaos and α = 3.82676 shows the intermittency chaos.

∂Epk−1,k∂wi,jk−1,k+ζ4p wi,j(m − 1) + noisei,j (m),(1)

noisei,j (m) = βi,j (m)(xi,j (m) − 0.5),(2)

The chaotic noise is generated by the logistic map
where α denotes the control parameter of chaos(Eq.(3)).
xi,j (m + 1) = αxi,j (m)(1 − xi,j (m),(3)3. Simulation Results
We consider the feedforward neural network produce
outputs x2 for input data x as one learning example.
The sampling range of the input data is [-1.0, 1.0]. The
learning rate and the inertia rate are ﬁxed as η = 0.02
and ζ = 0.002 respectively. The initial values of theweights are given between -1.0 and 1.0 at random. The
learning iterations is set to 10000 and 6 neurons areprepared in the hidden layer. The network structure
and learning example are shown in Fig. 1.
We investigate the learning eﬃciency by adding thechaotic noise into diﬀerent position of weight updateprocess ; a) Proposed network-1 (from input layer
to hidden layer only), b) Proposed network-2 (fromhidden layer to output layer only) and c) Proposed(a) α = 4.0.

(b) α = 3.82676.

Figure 2: Learning curve for amplitude noise β = 0.01.
Both ﬁgures show that the presence of noise during weight update process gives a better convergence
rate compared to the conventional method. From thesimulation result, we conﬁrm that when the chaotic
noise is applied to both sides of weight update process(Proposed network-3), the learning process gain better performance compared to other positions. Furthermore, the intermitency chaos shows a large diﬀerencein learning performance when the noise amplitude isset to 0.01 and the fully-developed chaos gives a slight
improvement only for the learning performance.

4. Conclusions
In this study, we proposed a new modiﬁed BP learning algorithm by adding chaotic noise into weight update process. By computer simulations, we conﬁrmedthat the addition of chaotic noise during weight update process can give a better convergence rate andfaster learning performance compared to the conventional BP algorithm.