平成18年度電気関係学会四国支部連合大会

17 - 21317

Durability of Aﬀordable Neural Networkfor Neuronal Death
Yoko Uwate, Yoshifumi Nishio and Ruedi Stoop(Tokushima University)(University/ETH Zurich)

In this study, we investigate the durability of the aﬀordable
neural network when some of the neurons in the hidden layer
are damaged, after the learning process.

2. Damages Neurons of Aﬀordable NN
In Refs. [1], we have proposed a network with aﬀordable
neurons in the hidden layer of the feedforward neural network
structure for eﬃcient BP learning. We introduced the aﬀordable neurons to reﬂect important properties of the brain.
We assume that some neurons in the hidden layer are damaged by some causes after the BP learning. The connections
to the output layer of the damaged neurons are cut as shownin Fig. 1. Namely, the damaged neurons do not operate. In
this situation, we investigate the performance of the network
when the learning data are inputted to the network.

damaged neurons. By comparison of two networks, the affordable neural networks gain better performance than theconventional neural network when some neurons are damaged. From these results, we conﬁrmed that the aﬀordable
neural networks can operate well even if some neurons aredamaged.
0.35
Affordable NN (9-1)
Affordable NN (9-2)
Affordable NN (9-3)
Conventional NN (9-0)

0.3
0.25
0.2

Eave

1. Introduction

0.15
0.1
0.05
00

1
2
Number of damaged neurons

After learning
Damagedneuron3

Figure 2: Eave when some neurons are damaged.
3.2. Position of Damaging Neurons
Next, we investigate the relationship between performanceand position of the damaging neurons. The simulated resultswhen the damaged neurons is only one are shown in Fig. 3.
From this ﬁgure, the conventional neural network is not good.
And the performance of the conventional neural network does
not depend on the position of the damaged neurons. On
the other hand, the aﬀordable neural network is almost verysmall when any neuron in the hidden layer is damaged.

DamagedneuronOutput layer

Input layer
Hidden layer

1

Figure 1: Damaging neurons.

Affordable NN (9-1)
Conventional NN (9-0)
0.1

We consider the feedforward neural network for the task toproduce output x2 for input x, as one learning example. We
investigate the total error between the output and the desired
target when some neurons are damaged after learning. We
deﬁne “Average Error Eave ” by the following equation.
Eave =P 
1  1

P

p=1

2



(tpi − opi )2 ) .

(1)

Eave

3. Simulated Results
0.01

0.001

0.0001

12

3

4
5
6
Position of damaged neuron

789Figure 3: Eave and position of damaged neurons.
4. Conclusions

In this study, we prepare 9 neurons in the hidden layer ofthe network and the number of the aﬀordable neurons is set
to 1 to 3. For comparison, we investigate the performanceof the conventional neural network without any aﬀordableneurons.

In this study, we conﬁrmed that the aﬀordable neural network keeps its good performance. It is obvious that the affordable neurons exert an important inﬂuence on durabilityof the network.

3.1. Eﬀect of Damaged Neurons

Reference

The simulation result is shown in Fig. 2. The Eave ofboth the aﬀordable neural network and the conventional neural network becomes worse by increasing the number of the[1] Y. Uwate and Y. Nishio, “Back Propagation Learningof Neural Networks with Chaotically-Selected Aﬀordable
Neurons,” ISCAS’05, pp. 1481-1484, May, 2005.