IEEE Workshop on Nonlinear Circuit Networks
December 14-15, 2012

Research of Pattern Recognition
by Multi-Layer Perceptron with Neurogenesis
Yuta Yokoyama, Chihiro Ikuta, Yoko Uwate and Yoshifumi Nishio
Department of Electrical and Electronic Engineering, Tokushima University
2-1 Minami-Josanjima, Tokushima-shi, Tokushima, 770-8506, Japan
Email: {yuta, ikuta, uwate, nishio}@ee.tokushima-u.ac.jp

Abstract— It was impossible to generate the new neuron in
the adult brain. Neurogenesis is that new neurons are generated
in the human brain. We focus on the characteristic of theneurogenesis with biologically. We apply the behavior of theneurogenesis to the artificial neural network. In the previousstudy, we have proposed artificial network model which wasapplied the neurogenesis to Multi-Layer Perceptron (MLP).
In this study, we investigate setting method of the connecting
weight of neurogenesis. By using computer simulations, weresearch the learning performance of the proposed MLPs by
setting of the parameter of the connecting weight of neurogenesis.

the learning and new neurons are generated in the human brain.
We have proposed artificial network model which was applied
the neurogenesis to Multi-Layer Perceptron (MLP) [5]. Then,the performance of the proposed network being submitted.
Figure 1 shows a structure of the proposed network.

I. I NTRODUCTION
It is said that we have about 10 billion neurons in the brain.
The network of the brain is formed by connecting of morethan one neuron. However, neurons had been considered tobe lost with age until several years ago. There is no neuralstem cell which makes a neuron in the brain of an adulthuman. In recent studies, some researchers reported that newneurons are generated in the dentate gyrus of hippocumpus[1]-[3]. This process is called “neurogenesis”. By utilizing
the neurogesis, some brain cells increase and the networkof within is substantial. It is known that the neurogenesis
improves memory, learning and thinking ability by combining
new neurons with biological neural network. We focus on thecharacteristic of the neurogenesis with biologically. In the previous study, we have proposed artificial network model which
was applied the neurogenesis to Recurrent Neural Network(RNN) [4] and Multi-Layer Perceptron (MLP) [5]. Therefore,
we have researched the performance of the proposed network.
The performance being submitted.
In this study, we focus on the values of the connecting
weight of the neurogenesis. We research that the connecting
weight of the neurogenesis change the some values.
II. MLP WITH N EUROGENESIS
Before, there is no neural stem cell which makes a neuron in
the brain of an adult human. It was impossible to generate thenew neuron. Therefore, the neurogenesis had been considered
to generate for period of growth. Thus, neurons had been considered to be lost with age until several years ago. However,some researchers reported that new neurons are generated in
the adult brain. This process is called “neurogenesis”. The
neurogenesis in the hippocumpus of the human brain wasdiscovered in the late 1990s by Erickson et al [1]-[3]. The
neurogenesis is included in the existing neural circuit by given

Fig. 1.

MLP with neurogenesis.

In this system, new neurons are generated in the hidden
layer during the learning process. At the same time, all theweights connecting to the generated neurons are newly set.
We explain two kinds of the neurogenesis. One is the periodic
neurogenesis. In the case of the periodic neurogenesis, newneurons generated at every 10 iterations during the learning
process. The other is the chaotic neurogenesis. In the caseof the chaotic neurogenesis, new neurons generated by using
the logistic map. Hasegawa et al. investigated solving abilitiesof the Hop-field NN with various surrogate noise, and they
concluded that the effects of the chaotic sequence for solving
optimization problems can be replaced by stochastic noisewith similar autocorrelation [6]. The updating function of thelogistic map is described by Eq. (5).
y(n + 1)= αy(n)(1 − y(n)).

(1)

We use that the parameter α =3.82740 and 4.0. The neurogenesis is occurred when the value of y takes the range between
0.6 to 0.7.

- 115 -TABLE I

III. S IMULATIONSP ERFORMANCE AS DIFFERENT CONNECTING WEIGHTS OF THE

We consider a pattern recognition, where 26 alphabets patterns in Fig. 2 are stored in the neural network for recognition.

NEUROGENESIS .
(a) The value of the neurogenesis’s connecting weight. (1) Random values.
(2) Copy of values of the next neuron. (3) Division of values of the next
neuron. (4) Constant values (positive value : 0.25). (5) Constant values(negative value : −0.25).

(a)Fig. 2.

Conv.
(1)
(2)
(3)
(4)
(5)

Input patterns.

We consider that the propose network is composed of threelayers. The number of neurons in the input layer is 35, and theoutput layer is 26. In the proposed MLPs, we set 10 neurons
in the hidden layer at the start of learning. Therefore, we set
the maximum number of the neurons in the hidden layer. TheMLPs learn 5000 iterations during the one trial. The learning
rate is η = 0.005 and initial values of the weights are between−0.5 and 0.5 at random. Moreover, we compare the learning
performance of the following five kinds of MLPs:
1) The conventional MLP
2) The MLP with random neurogenesis
3) The MLP with periodic neurogenesis
4) The MLP with intermittently chaotic neurogenesis(α=3.82740)
5) The MLP with fully chaotic neurogenesis (α=4.0)
We make a comparison between the performance of the conventional MLP and the proposed MLPs. Moreover, we carryout 100 trials with different initial weights of connections.
The performance of the proposed network being submitted.
In the results, we were able to obtain the performance of theproposed MLPs with 20 neurons as well or better than theconventional MLP with 30 neurons in the hidden layer. We
were able to obtain that the average of the processing timeof the proposed MLPs are faster than the conventional MLP
because used the small number of neurons.
We research the setting method of the connecting weight ofneurogenesis. Therefor, we research the learning performanceof the proposed MLPs by setting of the parameter of theconnecting weight of neurogenesis. We set 10 neurons in
the hidden layer at the start of learning process. During thelearning process, the network are set that the number ofneurons in the hidden layer increases until 20 neurons. We
compare the learning performance of the following five kindsof parameters of the neurogenesis’s connecting weight:
1) Random values2) Copy of values of the next neuron3) Division of values of the next neuron4) Constant value (positive value : 0.25)
5) Constant value (negative value : −0.25)The random value of connecting weights are between −0.5
and 0.5 at random. We show the average of the learning
performance and the standard deviation in Tab. I. Moreover,
we show that the conventional has 30 neurons in the hidden
layer.

(a)
Conv.
(1)
(2)
(3)
(4)
(5)

(1) The average of the learning performance.
Proposed network types
RandomPeriodic
Intermittently
0.00282
0.003070.00306
0.00288
0.003160.003140.00300
0.003130.003170.00305
0.008880.00937
0.00872
0.008610.008630.00824(2) The standard deviation.
Proposed network types
RandomPeriodic
Intermittently
0.001610.001610.001400.00160
0.001460.001180.001680.001030.001180.00101
0.00272
0.00280
0.00325
0.002450.002210.00235

Fully
0.00292
0.003130.00306
0.008830.00855

Fully
0.001210.001560.001120.002960.00238

From Table I (1), we can say that the learning performanceof the constance value is the worst in all networks. Namely,
we considered that the connecting weights of the neurogenesis
are not good the constant values. Moreover, we considered thatthe other networks are similar to the random values.
From Table I (2), we can see that the copy and division ofvalues of the next neuron are better than the random valueof neurogenesis’s connecting weights. Because, we consider
that the information of neurons is participated with each otherneuron. Namely, we were able to obtain the good performanceby the copy and division of values of the next neuron.
IV. C ONCLUSIONSIn this study, we investigated setting method of the connecting weight of neurogenesis.
We researched that connecting weights of the neurogenesischanged the some values. Then, we were able to obtain thegood performance of the standard deviation by copy anddivision of the next neuron.
R EFERENCES[1] S. Becker, J. M. Wojtowicz, “A Model of Hippocampal Neurogenesis in Memoryand Mood Disorders,” Cognitive Sciences, vol. 11, no. 2, pp. 70-76, 2007.
[2] R. A. Chambers, M. N. Potenza, R. E. Hoffman, W. Miranker, “Simulated Apotosis/Neurogenesis Regulates Learning and Memory Capabilities of Adaptive NeuralNetworks,” Neuropsychopharmacology, pp. 747-758, 2004.
[3] H. Satoi, H. Tomimoto, R. Ohtani, T. Kondo, M. Watanabe, N. Oka, I. Akiguchi,
S. Furuta, Y. Hirabayashi and T. Okazaki, “Astroglial Expression of Ceramide in
Alzheimer’s Disease Brains: A Role During Neuronal Apoptosis,” Neuroscience,
vol. 130, pp. 657-666, 2005.
[4] Y. Yokoyama, T. Shima, C. Ikuta, Y. Uwate and Y. Nishio, “Improvement of Learning Performance of Neural Network Using Neurogenesis,” Proceedings of RISPInternational Workshop on Nonlinear Circuits and Signal Processing (NCSP’12),
pp. 365-368, Mar. 2012.
[5] Y. Yokoyama, C. Ikuta, Y. Uwate and Y. Nishio, “Performance of Multi-Layer Perceptron with Neurogenesis,” Proceedings of International Symposium on Nonlinear
Theory and its Applications (NOLTA’12), pp.715 718, Oct. 2012.
[6] M Hasegawa, T Ikeguchi, T Matozaki and K Aihara, “An analysis on additive effectsof nonlinear dynamics for combinatorial optimization,” IEICE Trans. Fundamentals,
vol. E80-A, no. 1, pp. 206-213, 1997.

- 116 -