平成 28 年度電気関係学会四国支部連合大会 講演論文集 (2016 徳島大学)
2016 SHIKOKU-SECTION JOINT CONVENTION RECORD OF THE INSTITUTES OF ELECTRICAL AND RELATED ENGINEERS (TOKUSHIMA)

1-23
Feed-forward Neural Networks with Temporal Change
in Gradient of Sigmoid Functions
Shinsaburo KITTAKA
Yoko UWATE( Tokushima University)

1. Introduction

First, we search the learning rate ( a ) that has the smallest error. Table 1 shows the error with changing learning
rate from 0.1 to 1.0.

Neural network is constituted by some element that iscalled neuron. This network is based on animal brain. Withsimilarity brain, it is appropriate in recognize written characters and recognize voice. Features of neural network areparallel processing and learning. From these features, neural network is appropriate in recognize brain signal, too.
Because the learning of neural network can learn brain signal that has diﬀerences with each person and condition.
However, the recognition brain signal rate is 86 percent inother research. Recognition rate is important for making
widgets. For decreasing this rate, the challenge is making
similar output signal to perfect signal. Although, the error
can not become zero, because of learning method. So ourtask is reducing error close to zero.
In this study, we focus on the learning method of Feedforward neural network. Back propagation is the learning
method of Feed-forward neural network. We change sigmoid functions and improve back propagation.

Table 1: The error with changing aaave
aave

0.3
0.02974
0.8
0.021370.4
0.022610.9
0.021120.5
0.02144
1.00.02144

Figure 3: Conventional learning.

Table 2: The error of proposed method

(1)

aave

0.1
0.108160.2
0.094960.30.08314

0.40.01304

0.50.01106

aave

0.60.01465

0.70.01575

0.80.01514

0.90.01659

1.00.01690

In this simulation, we obtain very small error. Thismethod is eﬀective to improve back propagation. The simulation result of our proposed method is shown in Fig 4.

Figure 1 shows sigmoid functions with changing “k” andFig. 2 shows diﬀerential form of sigmoid functions.

Figure 4: Proposed learning.

Figure 1: Sigmoid functions Figure 2: Diﬀerencial formwith “k”.

0.2
0.02438
0.70.02079

Next, we simulate the proposed method and change timing of starting proposed method.
Table 2 shows the error of proposed method. We show
only starting proposed method since 300 iteration.

In this study, we change gradient of sigmoid functionsat iteration. Sigmoid functions is used in converting input
signal to output signal at learning of neural network. We
decrease the parameter “k” that is coeﬃcient added into
natural logarithm’s multiplier.
1
1 + exp(−kx)

0.10.028960.6
0.02144

We define as the learning rate = 0.7, because it has thesmallest error. We use this learning rate in the next simulation. The simulation result of conventional method isshown in Fig 3.

2. Proposed method

f (x) =Yoshifumi NISHIO

From Fig 4, the wave is lost since starting decreasing “k”.
This phenomenon means stabler learning than conventional
method.

of sigmoid functions.

We decrease “k” by 0.3 at starting proposed method anddecrease “k” by 0.001 at iteration.

4. Conclusions
Our learning method is changing gradient of sigmoid
functions by decreasing “k” at iteration for back propagation in feed-forward neural network. In these simulations,
we obtain smaller error than the conventional method.
However, if the “k” is small and starting decreasing time isearly, our method makes error very large. So, this method
requires attention to set parameters.
In the future work, we resolution that why the error becomes small with our method.

3. Simulation results
We divide Iris plants into 3 types. There are total of 150
data entries. The data records 4 attributes and 3 classifications. The 4 attributes are sepal length, sepal width, petal
length and petal width. The 3 types of iris flower are Iris
Setosa, Iris Versicolor and Iris Virginica. We define parameters as learning iteration = 1000 and the number of hidden
layer’s neurons = 4.

23