Investigation of Characteristics ofMulti-Layer Perceptron with Neurogenesis†

Yuta Yokoyama† , Yoko Uwate† and Yoshifumi Nishio†
Department of Electrical and Electronic Engineering, Tokushima University
2-1 Minami-Josanjima, Tokushima-shi, Tokushima, 770-8506, Japan
Email: {yuta, ikuta, uwate, nishio}@ee.tokushima-u.ac.jp
S UMMARY

It is said that there are about 10 billion neurons in the human’s brain. The network is formed by connecting of more than
one neuron. However, neurons had been considered to be lost with age until several years ago. It was impossible to generate
new neuron in the adult brain. This process is called “neurogenesis”. The neurogenesis in the hippocumpus of the human brain
was discovered in the late 1990s by Erickson et al [1]-[3]. Neurogenesis is that new neurons are generated in the human brain.
We focus on characteristics of the neurogenesis with biologically. In the previous study, we have proposed artificial networkmodel which was applied the neurogenesis to Recurrent Neural Network (RNN) [4] and Multi-Layer Perceptron (MLP) [5].
In this study, we investigate in more detail the influences of neurogenesis. We apply the behavior of neurogenesis to MultiLayer Perceptron (MLP) which is one of a feed-forward neural networks. Then, we consider characteristics of extinction andgeneration of neurons. We explain how to introduce generated neurons and extinct neurons. We use that the MLP is composedof three layers (one input, one hidden, and one output layer). In this network, we choose the neuron which is smallest action andnew neurons are generated there. At the same time, new neurons are generated in the hidden layer during the learning. Then,all the weights connecting to the generated neurons are newly set small random values. In this study, we assume the process
to generated neurons and connection to “neurogenesis.” After that, the connection weights are newly calculated. Figure 1 (b)shows a structure of the proposed network. We consider that the proposed network is composed of three layers. The numberof neurons in the input layer is 25, and the output layer is 3. Then, we set that the conventional MLP has 5 neurons in thehidden layer. Therefore, the proposed MLP are set that number of neurons in the hidden layer increases.
From Fig 1 (c), we compare the conventional MLP and the proposed MLP. We are able to obtain the good performance bygenerating new neurons in the hidden layer. We considered that the characteristic of neurogenesis can be applied well.

Fig. 1.

(a) conventional network, (b) proposed network, (c) result.

R EFERENCES[1] S. Becker, J. M. Wojtowicz, “A Model of Hippocampal Neurogenesis in Memory and Mood Disorders,” Cognitive Sciences, vol. 11, no. 2, pp. 70-76,
2007.
[2] R. A. Chambers, M. N. Potenza, R. E. Hoffman, W. Miranker, “Simulated Apotosis/Neurogenesis Regulates Learning and Memory Capabilities ofAdaptive Neural Networks,” Neuropsychopharmacology, pp. 747-758, 2004.
[3] H. Satoi, H. Tomimoto, R. Ohtani, T. Kondo, M. Watanabe, N. Oka, I. Akiguchi, S. Furuta, Y. Hirabayashi and T. Okazaki, “Astroglial Expression ofCeramide in Alzheimer’s Disease Brains: A Role During Neuronal Apoptosis,” Neuroscience, vol. 130, pp. 657-666, 2005.
[4] Y. Yokoyama, T. Shima, C. Ikuta, Y. Uwate and Y. Nishio, “Improvement of Learning Performance of Neural Network Using Neurogenesis,” Proceedingsof RISP International Workshop on Nonlinear Circuits and Signal Processing (NCSP’12), pp. 365-368, Mar. 2012.
[5] Y. Yokoyama, C. Ikuta, Y. Uwate and Y. Nishio, “Performance of Multi-Layer Perceptron with Neurogenesis,” Proceedings of International Symposiumon Nonlinear Theory and its Applications (NOLTA’12), pp.715 718, Oct. 2012.