平成24年度 電気関係学会四国支部連合大会

23

1-23

Investigate of Multi-Layer Perceptron with Neurogenesis
Yuta YOKOYAMA, Chihiro IKUTA, Yoko UWATE and Yoshifumi NISHIO(Tokushima University)

1. Introduction
Neurogenesis is that new neurons are generated in thehuman brain. By utilizing the neurogenesis, some brain
cells increase and the network of within is substantial. Theneurogenesis causes the improvement of memory, learning,
thinking ability, and so on by generation new neurons withbiological neural network.
In this study, we apply the behavior of neurogenesis tothe artiﬁcial neural network. In the proposed neural network, we use the Multi-Layer Perceptron (MLP) which isone of a feed-forward neural network. In order to conﬁrm the eﬃciency of neurogenesis, we investigate the performance of MLP with neurogenesis for learning severalalphabet patterns.

2. Proposed Network
MLP is a most famous feed-forward neural network. In
this study, we use the Back Propagation (BP). We consider that the proposed network is composed the threelayers (one input, one hidden, and one output layer) MLP.
In the proposed neural network, some new neurons are
generated in a hidden layer during the learning process.
We consider a pattern recognition, where 26 alphabets
patterns are fed the neural network for recognition. Figure 1 shows a structure of the proposed MLP. We used theinput patterns of alphabets A to Z as shown in Fig. 2.

k−1,kwhere wi,jis the weight between the ith neuron of thelayer k − 1 and the j the neuron of the layer k, and η isthe proportionality factor known as the learning rate.

3. Simulation Results
In this study, we consider a pattern recognition, where26 alphabets patterns are set the input patterns. The
number of neurons in the input layer and output layer are
35 and 26. Similarly, we set 10 neurons in the hidden
layer at the start of learning. The number of neurons inthe hidden layer is increased from 20 to 60 during thelearning. The learning time is set to m = 5000. The
learning rate is η = 0.005, and initial value of the weightare given between -0.5 and 0.5 at random. Moreover, we
compare the learning performance of ﬁve kinds MLPs:
1. The conventional MLP
2. The MLP with random neurogenesis
3. The MLP with periodic neurogenesis
4. The MLP with intermittency chaotic neurogenesis
5. The MLP with chaotic neurogenesis
The periodic neurogenesis generates the new neuron at
every 50 iteration during the learning process. In the caseof chaotic neurogenesis, the new neuron is generated byusing the logistic map.
We show an example of the learning performance of theMLPs in Fig. 3. In this example, the proposed MLPs
are set that the number of neurons in the hidden layer isincreased from 10 to 40 neurons during the learning.

Figure 1: Proposed network. Figure 2: Input patterns.
The updating rule of neuron and the sigmoid function
are described by Eqs. (1) and (2).
xi (t + 1)=f( n∑)wij (t)xj (t) − θ,(1)

jf (a)=1,
1 + e−a(2)where x is the input or output and w is the connection
weight parameter and θ is threshold.
The total error E of the network is described by Eq. (3).
1 ∑∑(tpi − opi )2 ,
2pE

=n(3)

p=1 i=1

where E is the error value, p is the number of the input
data, n is the number of the neurons in the output layer,
tpi is the value of the desired target data for the pth input
data, and opi is the value of the output data for the pthinput data. The connection weight is described by Eq. (4).
k−1,kk k−1∆p wij= ηpj
opi = −ηPowered by TCPDF (www.tcpdf.org)

∂Ep
k−1,k
∂wi,j,(4)Figure 3: Learning performance of each MLP.
From Fig. 3, we can see that the learning performance ofthe proposed MLPs are better than the conventional network. We consider that it was able to slip out of the localminima by increasing some neurons during the learning.

4. Conclusions
In this study, we applied the behavior of neurogenesis
to the MLP. In the proposed neural network, some newneurons were generated in a hidden layer by eﬀect of neurogenesis. We proposed 4 timing methods to introduce
neurogenesis. The proposed MLPs were able to obtained
the good results for the learning performance by the introducing the neurogenesis.