平成 29 年度電気関係学会四国支部連合大会 講演論文集 (2017 愛媛大学)
2017 SHIKOKU-SECTION JOINT CONVENTION RECORD OF THE INSTITUTES OF ELECTRICAL AND RELATED ENGINEERS (EHIME)

15-5
Feedforward Neural Networks
with Threshold for Avoiding Over-Training
Akihiro ITAGAKI1 , Ryuta YOSHIMURA1 , Yoko UWATE1 and Yoshifumi NISHIO1(1 Tokushima University)
1. Introduction

hback and yback are back probagation. ε is the rate of decay
which is 0.01. The initial value of the weights is chosen byrandom from 0.01 to 0.04.

Neural Network is a model imitating the human brain.
It is widely used in various areas. Feedforward back propagation which is well known to solve social problem is oneof the Neural Networks. When learning is repeated for all
training data, over learning occurs. We keep network fromover training by setting the threshold. If the error value
falls below the threshold, training data would not be used
for learning.

3. Simulation result
We define as the number of input layer = 4, the numberof hidden layer = 4, the number of layer = 3, the learning
loops = 100000, training data sets = 5 and test data sets
=48. Iris data set is used for training data. The test data islarger than the training data because it causes over training.
The threshold is set to 7 patterns and error value is shownin Table 1. The error in this is an error in the test data.
The error value is the average of 20 simulations. In caseof error < 0.3, the result is the smallest. However, in caseof error < 1.0, error value becomes larger than when thethreshold value is not set. If you set the threshold too high,the learning loops will decrease significantly. As a result,when we set the threshold error < 0.3, the best result isobtained.

2. Proposed method
Proposed system that is used in this study is shown in
Fig.1. The right arrow indicates the propagation of theinput signal. The reverse arrow indicates the back propagation in Fig.1. In this study, the error value in the training
data is not used for learning below the set value. Thereby
it can prevent the problem of over training. This proposedsystem can reduce error value.

Table 1: Error value
thresholdno thresholderror < 0.01
error < 0.05error < 0.1error < 0.3error < 0.5
error < 1.0error value
0.445
0.439
0.409
0.393
0.391
0.428
0.466

4. Conclusion
As the simulation results, we found that setting thethreshold reduces over training. Therefore the threshold
reduces the error value. Especially in case of error < 0.1
and error < 0.3, the proposed method obtains good results.
In the future work, we would like to obtain better results.
Therefore, we would like to consider a new proposed method
such as adding noise to data. Furthermore, We think that
we will try to reduce error value by deep learning which
increased the hidden layer.

Figure 1: Feedforward Newral Network.
 h= y=1(−1+e

1 + e(−∑(w1 x))

1∑(1)(w2 h))

Reference[1]Isao Taguchi and Yasuo sugai, “The Error Territory
in the Functional Learning of the Three-Layer Networks”,
Keiai University International research. vol. 10, pp.63-67,
2002.
[2]Jacques de Villiersand Etienne Barnard,“Backpropagation Neural Nets with One and Two HiddenLayers”, IEEE Transactions on Neural Networks. vol. 4.
no.1 January 1992, pp. 136-141, 1992.

Equation (1) is the sigmoid functions. It is used in theoutput of each neuron and calculation in back propagation.
h means propagation. x means input. y means output. w
means weights between neurons.

{yback = (y − ty)(1 − y)yhback ={∑wyback (1 − h)hw1 (l+1) = w1 l − ε xhback
w2 (l+1) = w2 l − ε hyback

(2)

(3)

168