平成 29 年度電気関係学会四国支部連合大会 講演論文集 (2017 愛媛大学)
2017 SHIKOKU-SECTION JOINT CONVENTION RECORD OF THE INSTITUTES OF ELECTRICAL AND RELATED ENGINEERS (EHIME)

15-7
Learning Method Adding Noise to Inertia Termof Feedforward Neural Network
Yusuke NOMURA Kazuki NAGAO Yoko UWATE Yoshifumi NISHIO( Tokushima University)
1. Introduction{Neural Network (NN) is imitated the working of the human nervous system. NN attracts attention because it isspecialized in pattern recognition and image recognitionand so on. However, it takes time to process pattern recognition and image recognition and so on.
To solve this problem we experiment to improve learning
accuracy and to reduce learning loops. In this study, wepropose Feed Forward NN(FFNN) with inertia term having Gaussian distribution noise. Conventional system andproposed system are compared.

{w1 (l+1) = w1 l − ε xhback
w2 (l+1) = w2 l − ε hyback(3)

w1 (l) = αw1
w2 (l) = αw2(4)

ε is the rate of decay which is 0.01. h means propagation.
w means weights between neurons. The initial value of theweights is chosen by random from 0.01 to 0.04. α is inertia
coeﬃcient.
Equation (5) is proposed systems equation in this study.

2. Proposed method
Conventional and proposed systems that is used in thisstudy are shown in Fig. 1. Arrows of left to right indicate propagation of input signal and right to left indicate
learning of back propagation. We propose Feed Forward
NN (FFNN) with inertia term having Gaussian distribution
noise. In this study, we use FFNN about 4 neurons input
layer, 4 neurons hidden layer and 3 neurons output layer.
We use Gaussian distribution (0, 0.01) for noise and add itto Equation. (4). In addtion, the putting place of noise ischanged. We experiment 4 patterns about conventonal system, (input layer-hidden layer) = IH, (hidden layer-outputlayer) = HO and (input layer-hidden layer, hidden layer output layer) = IHHO.

{w1 (l) = αw1 + (αw1 +β)
w2 (l) = αw2 + (αw2 +β)(5)β is Gaussian distribution noise.

3. Simulation result
We define as the number of input layer = 4, the numberof hidden layer = 4, the number of output layer = 3, thelearning loops = 50000 and the number of learning data sets
= 150. We used Iris data set as learning data. We simulate
proposed method five times and calculate the average error
rate and minimum error rate. Table 1 shows average error
rate and minimum error rate for all kinds of patterns.

Table 1: Average and minimum error rate.
ConventionalIH
HOIHHO

4. Conclusion

Equations (1) to (4) are general FFNN’s propagation
equations, back propagation equations and weight correction equations.

 y={We comfirm that it is good to add noise to the inertia term. However, when we use the proposed method atIHHO, its measurement became impossible. The reason forthis is considered to be caused by over training. As a causeof over training, there are lack of learning data and a lot of
the propotion of biased data in learning data. In this study,
we consider to make a lot of the propotion of biased data
in learning data cause over trainig.
In the future,we developed a model to suppress overtraining and we would like to combine to proposed model.
We use a variety of noise and want to obtain better results.

11 + e(−1 + e(−∑(w1 x))

1∑hback =∑(1)(w2 h))

yback = (y − ty)(1 − y)yw2 yback (1 − h)hminimum error rate
0.08530.02340.0261
unmeasurable

We comfirm that average error rate and minimum error
rate of the proposed method are better than the conventional method from Table 1. Moreover, we understand that
average error rate and minimum error rate of IH is better
than HO from Table 1. In case of IHHO, we could notmeasure the average and minimum error rate.

Figure 1: Schematic diagram of FFNN.

 h=

average error rate
0.09450.03550.0534
unmeasurable(2)170