平成 26 年度電気関係学会四国支部連合大会 講演論文集 （2014 徳島大学）
2014 SHIKOKU-SECTION JOINT CONVENTION RECORD OF THE INSTITUTES OF ELECTRICAL AND RELATED ENGINEERS (TOKUSHIMA)

1-18
Cellular Neural Networks with Update Template
by Using Reinforcement Learning
Kazushige NATSUNO, Yoko UWATE and Yoshifumi NISHIO(Tokushima University)

1. Introduction
Binary template:

In a general way, a conventional CNN can be easy toimplement as a hardware by simple structure. However,the conventional CNN is restricted to the performance byspatially-uniform template. Therefore, we propose CNN
with update template by using reinforcement learning. Inthe proposed CNN, the templates are updated by neighboring output values.
From some simulation results, we confirm that the proposed CNN is eﬀective for image processing.

[A=010121010], B = T = 0.

(6)

2. Proposed CNN
In this section, we explain the algorithm of the proposedCNN. In the proposed CNN, the template of all cells arechanged at every iterations by reinforcement learning. Thelearning steps in proposed CNN are described as follows.
Step 1 : State and output values are updated according toEqs. (1) and (2) as follow.
State equation :
dvxij
dt=

∑i+r

−vxij +

∑i+r+

∑j+r

∑j+r

k=i−r

k=i−r

l=j−r

(|i−k|≤1,l=j−r

A(i,j;k,l) vxkl (t)

(a)

(b)

(c)

(d)

B(i,j;k,l) vukl (t)+T,|j−l|≤1).

(1)

Output equation :1 (|vvyij (t)= 2
xij (t)+1|−|vxij (t)−1|).

(2)

Step 2 : The output values of the center cell and neighbor cells are compared. The comparison equation for thecell(i, j) is described Eq. (3).
Comparison Equation :neighbo
centerD(i,j;k,l)=|vy(i,j)−vy(k,l)|.

Figure 1:

Simulation results. (a) Input image, (b) Conventional CNN, (c) Proposed CNN(Similar cell), (d) Proposed
CNN(Diﬀerent cell).

(3)

Step 3 : Among the eight calculated values of D(i, j; k, l),
the diﬀerence values of cell are ranked based on the smallvalue, respectively. In addition, templates of similar anddiﬀerent cells are updated according to Eqs. (3) and (4) asfollow.
TD-error :presently}−ainitial.
(i,j;k,l)(i,j;k,l)

T D−error={Delta(i,j;k,l)+γ∗aIn Fig. 1(a), the buildings of the background are the indistinct and the detail portions. Figure 1(b) shows the simulation result by the conventional CNN. One the other hand,Fig 1(c) and (d) show the simulation results by using theproposed CNN. In the case of using the conventional CNN,
the indistinct portions are not detected. In addition, thedetail portions are blacked out in Fig. 1(b). In contrast, inFigs. 1(d), the indistinct portions can be detected. Also,in the case of updated template at similar value, the detailportions are blacked out in Fig. 1(c). On the other hand,in the case of updated template of the diﬀerent values, thedetail portions are shown in Fig. 1(d).
From the simulation results, in the proposed CNN, theoutput values are changed by update template of selected
cell. Therefore, we can say that the proposed CNN is more
eﬀective than the conventional CNN.

(4)

Updated equation :aupdated
presently=a−α∗(T D−error).
(i,j;k,l)(i,j;k,l)(5)

where a(i, j; k, l), α and γ represent the template, a learning rate (0 ≤ α ≤ 1) and a reduction rate (0 ≤ γ ≤ 1),respectively.
Step 4 : The step 1 to 3 are repeated every 0.005 [τ ].

4. Conclusions

3. Simulation result

In this study, we proposed CNN with update template byusing reinforcement learning. In the proposed CNN, templates are updated by neighboring output values.
From the simulation results, in the proposed CNN, theoutput values is changed by update template of selectedcell. Therefore, we can say that the proposed CNN is more
eﬀective than the conventional CNN.

In this section, we show simulation results using the proposed CNN. In this simulation, the parameters of the learning rate and the reduction rate are set to α = 0.5 andγ = 0.5, respectively. Also, templates and threshold areassigned as follows.

18