2010 International Symposium on Nonlinear Theory and its Applications
NOLTA2010, Krakow, Poland, September 5-8, 2010

Characteristics of Cellular Neural Networks with Dynamic Template forMotion Pictures
Masakazu KAWAHARA† , Yoko UWATE† and Yoshifumi NISHIO†
Email:{kawahara, uwate, nishio}@ee.tokushima-u.ac.jp†Dept. Electrical and Electronic Eng.,
Tokushima University
Abstract—In our previous research, cellular neural networks with dynamic template (D-CNN) have been proposed. In D-CNN, the wiring weights of template are dynamically changed at each update by learning. In this study,
we investigate the characteristics of the update template inD-CNN when input images are changed during D-CNN
process. We express the variation of the update templatethrough the gray scale. We also calculate the increasing or
decreasing ratios of element in the update template. Fromthese obtained results, we confirmed that D-CNN is effective for motion pictures.

put image. From the investigation of the value of the updated template, we confirmed the characteristic of updated
template in D-CNN by changing input images. The rest of
this paper is structured as follows. In the Sec. 2, we review
the basic of the standard CNN. In the Sec. 3, we explain thealgorithm of the proposed D-CNN. In the Sec. 4, we show
the characteristic of the updated template in simple binary
images using D-CNN. In the Sec. 5, we show the characteristic of the updated template in real gray scale images
using D-CNN. The Section 6 concludes the article.
2. Cellular Neural Networks [1]1. Introduction
Cellular neural networks (CNN) were proposed by Chuaand Yang in 1988 [1]. The idea of CNN was inspired fromthe architecture of the cellular automata and the neural networks. Unlike the original neural networks, the CNN has
local connectivity property. Wiring weights of the cells are
established by parameters called the template. The performance of the CNN is decided by the template. Also, theCNN has been successfully used for various high-speed
parallel signals processing applications such as image processing application [2][3]. Usually, the templates of all
the cells in the CNN are identical and those values do notchange during the processing. This is good for implementation but restrict the performance, namely the conventional
CNN can not perform image processing based on the local
features of input images.
In the previous study, we have proposed cellular neuralnetworks with dynamic template (D-CNN) [4]. In D-CNN,
template is dynamically changed at each update by learning. This learning method is inspired from the rank orderlearning. The updated template depends on the value ofcells. From the simulation results of the previous study,
we confirmed that the converged value of each cell is divided to two or three values. Also, convergence process ismuch more rapid than that of the conventional CNN. Then,
we investigate update template in D-CNN for motion pictures [5]. However, the mechanism of D-CNN has not been
made clear. In this study, we investigate the characteristicof the updated template using CNN with dynamic templatein detail. We also set to threshold value initial template forcomparison of the updated template. We expect the appearance of characteristics in updated template by changing in-In this section, we explain the basic structure of theCNN. The CNN has M by N processing unit circuits calledcells. Cells are arranged in a reticular pattern to M line Nrow. We represent a cell C(i, j) using a variable i which
denotes vertical position and a variable j which denotes
horizontal position. The cell contains linear and nonlinear circuit elements. The CNN is an array of cells. Eachcell is connected to only its neighboring cells according toa template. Usually, the template is the same for all cells
except for boundary cells. The CNN has the features oftime continuity, spatial discreteness, nonlinearity and parallel processing capability
State equation:
dv xi jdt=−v xi j +j+ri+r X
XA(i, j;k,l) vykl (t)k=i−r l= j−r+j+r
i+r XX

B(i, j;k,l) vukl (t) + I.

(1)

1(|v xi j (t) + 1| − |v xi j (t) − 1|).
2

(2)

k=i−r l= j−r

Output equation:
vyi j (t) =where v x , vy and vu represent a state, an output and an inputof cell, respectively. In the equation (1), A is the feedback
template and B is the control template. These and bias I
are collectively called general template. In this equation,
the control template depends on input value.

- 23 -3. CNN with dynamic template
In this section, we explain the algorithm of D-CNN. In
our research, we change input images when a certain calculation times comes. In our D-CNN, the templates are updated at every iterations by rank order learning. The learning steps in our D-CNN are described as follows.
STEP 1: The state values and the output values of all thecells in D-CNN are updated according to the discretizedmodel of Eqs. (1) and (2).
STEP 2: Calculate the comparison of the output value ofeach cell with the one-step-past outputs of the cell and its
neighbor cells. The comparison equation for the cell (i, j)
is described Eq. (3).
Comparison Equation:
past
now
Di f (i, j; k, l) = |vy,(i,j) − vy,(k,l) |.

In this study, we decide Number o f calculationmax inEqs. (5) and (6) to be set to 10. Namely, the learning rates
of “winner” and “second” are changed until 10 calculations. Then, after Number o f calculationmax becomes over
10, the learning rates of “winner” and “second” become 0
and the templates are not updated. By using the learning
rate, the elements of the template are updated according to
the following update equation.
Update Equation:
past
now
now
aupdatedwinner = awinner − R1 (vy,(i, j) − vy,(i, j) ).

(7)

past
now
now
aupdatedsecond = a second − R2 (vy,(i, j) − vy,(i, j) ).

(8)

R1 and R2 decrease according to the Eqs. (5) and (6). The
initial learning rates are given as follows.
Initial Learning rate:(3)

STEP 3: Among the 9 calculated values of Di f (i, j; k, l),the cells with the smallest and the second smallest values
are defined as “winner” and “second”, respectively. In ourupdate algorithm, we change the learning rate in two elements. By this step, we find the position of cells with thenearest and the second nearest values to the corresponding
cell (i, j).
STEP 4: Update the elements of the template corresponding to the positions of the “winner” and the “second”. Notethat in our proposed learning algorithm only two elements
are updated. The update method and the update function
are described as follows.
Update Method:
Assume that the template before update is given as
Eq. (4).
T emplatenow : now a11anow
anow
1213 
 nownow
now
now  ,aaa
A(i, j) =  21
2223  nownow a31
anow
a32
33

 now

 b11bnow
bnow
1213 
 nownow
now  ,bnowbB(i, j) =  b21
2223 

 nownow b31bnow
b32
33
now
now
I(i,.
(4)j) = I
For example, we consider the case that the “winner” is (i, j)
and the “second” is (i − 1, j − 1). In that case, only anow
22 ,nownow,andinEq.
(4)areupdated.
Thethresholdbnow
ab
22
11
11
value I is not updated in our learning method.
In our update algorithm, we change the learning rate in
two elements. The learning rates of the “winner” and the“second” are shown as follows.
Learning rate:!
Number o f calculation.
(5)
R1 = R10 1 −Number o f calculationmax!
Number o f calculation
R2 = R20 1 −.
(6)
Number o f calculationmax

WinnerS econd::R10 (0 ≤ R10 ≤ 0.1).
R20 = R10 /4.

(9)
(10)

After the update using Eqs. (7) and (8), the updated template is shown as follows. In Eq. (11), aupdated
and aupdated
11
22
are the updated values. Also, bupdated
and bupdated
are up11
22
dated similarly.
T emplateupdated : updated a11anowanow
1213  now
updatednow Aupdated= ,aaa 21(i, j)
22
23 now
now anow
aa313233
 updated
now  b11bnowb1213
updatednow  ,Bupdated=  bnowbb(i, j)
2223  21now
now
now b31b32
b33
updatedI(i,= I now .
j)

(11)

STEP 5: The steps from 1 to 4 are repeated.
These learning steps inspired from the rank order learning.
4. Investigation Results
In this section, we show the investigation results for twotypes of motion pictures. We express the variation of theupdated template in D-CNN through the graduation value.
In the first step of this investigation, an initial template isset to D-CNN. The elements of the initial template are updated by using updated method in the previous section. The
initial template used in this study is described as follows.
Initial Template: 0.07 0.1 0.07  0 0 0 
 , I = 0. (6)


0.1
0.32
0.1
000A =  , B = 000

0.07

0.1

0.07

4.1. The Updated Template in Binary images
Firstly, we investigate updated template for simple binary image. We change 4 input images every 10 [τ] like amotion picture.

- 24 -

(a)

(b)

(a)

(b)

Figure 2: The graduation of updated template. (a) Theupdate template after processing for the Input image 3.
(b) The final update template.

(c)number of the update template is varied by changing input
image.

(d)Figure 1: Motion pictures 1. (a) Input image 1. (b) Input
image 2. (c) Input image 3. (d) Input image 4.

Figure 1 shows binary input images for motion pictures.
In Fig. 1, we recognize the black object appeared andmoved up to down through Figs. 1(a) to (d). By changing these input images, we investigate the characteristic of
the updated templates.
In this investigation, we show that how to update the dynamical template. Namely, we calculate the difference ofelements in updated template from the initial template. We
set to the threshold value in updated template. The threshold value of A and B templates are described as follows.
Threshold value of template:
T hA (i, j) =3 X3Xai j(12)

bi j .

(13)

i=1 j=1

T hB (i, j) =3 X
3X
i=1 j=1

Namely, the threshold value is summed all of the elements in the each template. From our update algorithm,
the rate of update is same rate in T hA and T hB .
Figure 2 shows the variation of the update templatethrough the gray scale. In Fig. 2, if the average of update
template is over T h, the pixels near to black. While, if theaverage of update template is under T h, the pixels near towhite. From these simulation results, we can confirm thatthe dynamical templates of the changing area from the previous image is more updated than the constant image area.
Figure 3 shows the percentage of updated template
whether the value of all the elements of updated template
under T h and over T h. In these graphs, we can say that the4.2. The Updated Template in Gray Scale Images
Next, we investigate updated template for gray scale image. Similar to the previous investigation, we change 4 input images every 10 [τ] like a motion pictures.
Figure 4 shows the input images as motion pictures. We
confirm the doll is appeared and gradually move left toright through Figs. 3(a) to (d). Using these input images,
we investigate the characteristic of update template in DCNN.
Figure 5 shows the variation of the update templatethrough the graduation value with gray scale. In Fig. 5,
the area of the edge is most changed by changing input image. From these results, we can say that the edge of new
object is more updated by inputted next input image.
Next, we investigate the process of the number of the updated template. Figure 6 shows the the number of the updated template is over T h and under T h. In Fig. 6, the number of template over T h decreases gradually. On the otherhand, the number of template under T h increases gradually. From these results, we confirm that the average of thechanging rate in updated template is varied widely when
the next input image is inputted. Even if next input image
is inputted, the updated template influence of before input
image. Therefore, we can say that our proposed CNN iseffective for the motion picture processing.
5. Conclusions
In this study, we have investigated the characteristics of
the update template in D-CNN when input images werechanged during D-CNN process. In D-CNN, the template
was changed by rank order learning. We have expressed thevariation of the update template through the gray scale. We
also calculate the increasing or decreasing ratios of elementin the update template. From instigation results, we haveconfirmed that the D-CNN is effective for the applicationof motion pictures.

- 25 -(a)(b)

Figure 5: The graduation of the updated template. (a) The
updated template during simulation. (b) The final updatedtemplate.

(a)

(b)
Figure 3: The number of updated template in Fig. 1.
(a) The percentage of template where under T h(i, j) . (b) The
percentage of template where over T h(i, j) .

Figure 6: The number of updated template in Fig. 5.

References[1] L. O. Chua and L. Yang, “Cellular Neural Networks:Theory,” IEEE Trans. Circuits Syst., vol. 32,
pp. 1257-1272, Oct. 1988.
[2] F. Dirk and T. Ronald, “Coding of Binary Image Data
using Cellular Neural Networks and Iterative Annealing,” Proc. of ECCTD’03, vol. 1, pp. 229-232, Sep.
2003.
(a)

(b)

[3] M. Namba and Z. Zhang, “Cellular Neural Network for
Associative Memory and Its Application to Braille Image Recognition,” Proc. of IJCNN’06, pp. 4716-4721,
Jul. 2006.
[4] M. Kawahara, T. Inoue and Y. Nishio, “Cellular Neural
Network with Dynamic Template and its Output Characteristics,” Proc. of IJCNN’09, pp. 1552-1558, Jun.
2009

(c)

(d)

Figure 4: Input images. (a) Input image1. (b) Input image2(after 10 [τ]). (c) Input image3 (after 10 [τ]). (d) Output
image before input image is changed.

[5] M. Kawahara, T.Inoue and Y. Nishio, “Image Processing Application Using CNN with Dynamic Template”
Proc. of CNNA’10, pp. 41-46, Feb. 2010.

- 26 -