2011 International Workshop on Nonlinear Circuits,Communication and Signal Processing
NCSP'11, Tianjin, China, March 1-3, 2011

Investigation of Output Characteristics of Cellular Neural
Networks with Dynamic Template
Masakazu KAWAHARA† , Yoko UWATE† and Yoshifumi NISHIO††Tokushima University
2-1 Minami-Josanjima, Tokushima, Japan
Phone:+81-88-656-7470,FAX:+81-88-656-7471
Email: {kawahara, uwate, nishio}@ee.tokushima-u.ac.jp

Abstract
In the previous study, we have proposed CNN with dynamic template (D-CNN). In D-CNN, template is dynamically changed at each update of learning. In this study, weconsider the two types of update equations for the dynamictemplate. We investigate the characteristics of the output
value of D-CNN by using two update equations.

mechanism of D-CNN, we focus on the output values of theD-CNN after image processing.
The rest of this paper is structured as follows. In the Sec. 2,
we show the algorithm of the proposed D-CNN. In the Sec. 3,
simulation results using the proposed D-CNN are shown. The
Sec. 4 concludes the article.

2. Cellular Neural Networks with Dynamic Template [2]1. Introduction
Cellular Neural Networks (CNN) were proposed by Chuaand Yang in 1988 [1]. The idea of CNN was inspired from thearchitecture of the cellular automata and the neural networks.
Unlike the original neural networks, the CNN has local connectivity property. Wiring weights of the cells are establishedby parameters called the template. The performance of theCNN is decided by the template. Also, the CNN has beensuccessfully used for various high-speed parallel signals processing applications Usually, the templates of all the cells inthe CNN are identical and those values do not change during
the processing. This is good for implementation however restrict the performance, namely the conventional CNN can not
perform image processing based on the local features of input
images.
In the previous study, we have proposed CNN with dynamic template (D-CNN) [2]. In D-CNN, template is dynamically changed at each update by learning. This learning
method is inspired from the rank order learning. The updated
template depends on the output value of cells. From the simulation results of the previous study, we have confirmed thatthe converged value of each cell is divided to two or three values. Then, we have investigated update template in D-CNN
for motion pictures [3][4].
In this study, we investigate the output characteristics ofD-CNN for four types of motion pictures and compare theperformance of D-CNN for extracting moving object by using
two types of update equations. In order to make clear theIn this section, we explain the algorithm of D-CNN. The
input images are changed when a certain calculation timecomes. In this proposed D-CNN, the templates are updated atevery iterations by rank order learning.
The learning steps in our D-CNN are described as follows.
STEP 1: The state values and the output values of all the cellsin D-CNN are updated according to state equation and outputequation.
State equation:
dv xi jdt=−v xi j +j+ri+r ∑∑A(i, j;k,l) vykl (t)k=i−r l= j−r+j+r
i+r ∑∑

B(i, j;k,l) vukl (t) + I.

(1)

1(|v xi j (t) + 1| − |v xi j (t) − 1|).
2

(2)

k=i−r l= j−r

Output equation:
vyi j (t) =STEP 2: Calculate the comparison of the output value of eachcell with the one-step-past outputs of the cell and its neighborcells. The comparison equation for the cell (i, j) is describedby Eq. (3).
Comparison Equation:- 60 -pastnow
Di f (i, j; k, l) = |vy,(i,j) − vy,(k,l) |.

(3)

STEP 3: Among the 9 calculated values of Di f (i, j; k, l), thecells with the smallest and the second smallest values are defined as “winner” and “second”, respectively. In our update
algorithm, we change the learning rate in the two elements.
By this step, we find the position of cells with the nearest andthe second nearest values to the corresponding cell (i, j).
STEP 4: Update the elements of the template corresponding to the positions of the “winner” and the “second”. Notethat in our proposed learning algorithm only two elements
are updated. The update method and the update function aredescribed as follows.
Update Method:
Assume that the template before update is given as Eq. (4).
T emplatenow : now a11anowanow
1213 
 anow anow anow  ,Anow(i, j) = 
212223
 now

a31anowanow
32
33
 now
 b11bnowbnow
1213  bnow bnow bnow  ,Bnow(i, j) = 
21
2223

 nowb31
bnowbnow
32
33
now
now
I(i,.
(4)j) = I
For example, we consider the case that the “winner” is C(i, j)now
and the “second” is C(i−1, j−1) . In that case, only anow
22 , b22 ,now
now
a11 and b11 in Eq. (4) are updated. The threshold value I is
not updated in our learning method.

rates of “winner” and “second” become 0 and the templates
are not updated. By using the learning rate, the elements ofthe template are updated.
In this study we define two types update equations(“Type1” and “Type2”) of D-CNN as follows.
Update Equation (T ype1):
past
now
now
aupdatedwinner = awinner + R1 (vy,(i, j) − vy,(i, j) ).

(7)

past
now
now
aupdatedsecond = a second + R2 (vy,(i, j) − vy,(i, j) ).

(8)Update Equation (T ype2):
past
now
now
aupdatedwinner = awinner − R1 (vy,(i, j) − vy,(i, j) ).

(9)

past
now
now
aupdatedsecond = a second − R2 (vy,(i, j) − vy,(i, j) ).

(10)

Where, R1 and R2 decrease according to the Eqs. (5) and(6). In the case of Type1, the amount of element change issummed to the current element value. While, in the case ofType2, the amount of element change is took away from thecurrent element value.
The initial learning rates are given as follows.
Initial Learning rate:Winner: R10 (0 ≤ R10 ≤ 0.1).

(11)

S econd: R20 = R10 /4.

(12)After the update using Eqs. (7) and (8) or Eqs. (9) and(10), the updated template is shown as follows. In Eq. (13),aupdated
and aupdated
are the updated values. Also, bupdated
and
11
22
11
updatedb22
are updated similarly.

Figure 1: Decision of updated elements of template in DCNN.

T emplateupdated : updated a11
anow
anow
1213
 now
updatednow Aupdated= ,aaa 21(i, j)
2223 nownow anowaa3132
33
 updatednow
now  b11b12b13 

Bupdated=  bnow ,bupdatedbnow(i, j)
212223 now
now b31bnow
b32
33

In our update algorithm, we change the learning rate in twoelements. The learning rates of the “winner” and the “second”
updatedI(i,= I now .
(13)j)
are shown as follows.
Learning rate:
STEP 5: The steps from 1 to 4 are repeated.
)(
Number o f calculation.
R1 = R10 1 −(5)
Number o f calculationmax)(
3. Simulation ResultsNumber o f calculation.
R2 = R20 1 −(6)
Number o f calculationmax
In this section, we show the simulation results by using two
The parameters of R10 and R20 are set to 0.1 and 0.025, re- types update equations (“Type1” and “Type2”) of D-CNN.
spectively. And Number o f calculationmax in Eqs. (5) and Figure 2 to 5 show the four types input images as motion(6) to be set to 10. Namely, the learning rates of “winner” pictures. We can recognize the position of object is changed
and “second” are changed until 10 calculations. Then, af- through the four figures.
ter Number o f calculationmax becomes over 10, the learning
For simulations, an initial template is set as follows.

- 61 -Initial Template: 0 0 0 
 0.07 0.1 0.07 
A =  0 0 0  , B =  0.1 0.32 0.1  , I = 0.
0.07 0.1 0.07
0 0 0

(a)

(b)

(c)

(14)
(a)
(b)
(c)
(d)
Figure 5: Input images. (a) Input image 1. (b) Input image 2.
(c) Input image 3. (d) input image 4.

(d)Figure 2: Input images. (a) Input image 1. (b) Input image 2.
(c) Input image 3. (d) input image 4.
(a)

(b)

Figure 6: Output images for Figs. 2. (a) Output image using“Type1”. (b) Output image using “Type2”.

(a)
(b)
(c)
(d)
Figure 3: Input images. (a) Input image 1. (b) Input image 2.
(c) Input image 3. (d) input image 4.

(a)

(b)

Figure 7: Output images for Fig. 3. (a) Output image using“Type1” in Fig. 3. (b) Output image using “Type2” in Fig. 3.
(a)(b)(c)(d)
Figure 4: Input images. (a) Input image 1. (b) Input image 2.
(c) Input image 3. (d) input image 4.
Figures 6 to 9 show the output images for the input images
in Figs. 2 to 5. From these results, we can say that the output
images have diﬀerent characteristics depending on the two
update equations.
Next, we investigate the distribution of the output values
in output images. We divided output image to four area likeFig. 10.
As an example, the distributions of output value in A areaof Figs. 6 to 9 are summarized in Tab. 1. In the results ofFigs. 6 and 8, we can recognize the value of background isconcentrate on −0.6 to −0.4. While, we can recognize thevalue of background is concentrate on 0.4 to 0.6 in the resultsof Figs. 7 and 9. Furthermore, we can confirm that the distribution of the output values by using “Type2” equation is
spread widely than the case of “Type1”.
Next, we propose the evaluation method how clear movingobject is extracted by using two update equations. We expectthat if the number of pixel which is belong to far value form

(a)

(b)

Figure 8: Output images for Fig. 4. (a) Output image using“Type1”. (b) Output image using “Type2”.

(a)

(b)

Figure 9: Output images for Fig. 5. (a) Output image using“Type1” in Fig. 5. (b) Output image using “Type2” in Fig. 5.

- 62 -The value of pixel−1.0 ∼ −0.8
−0.8 ∼ −0.6
−0.6 ∼ −0.4
−0.4 ∼ −0.2
−0.2 ∼ 00 ∼ 0.20.2 ∼ 0.40.4 ∼ 0.60.6 ∼ 0.80.8 ∼ 1.0

Table 1: Distribution of output image in A area.
The number of pixel
Figure 6
Figure 7
Figure 8
Type1 Type2 Type1 Type2 Type1 Type2739
4109385
1672
2036
87
1035158
10412338 13669
42313007 12853
161166472563
970
112
16849
806
53036
161045502
59027153629
43703911791 12364
15400352
38
118002870
122182

Figure 9
Type1 Type200002247554135
186
2138
446
12735 14219
71703
152
1between “Type1” and “Type2”.
Table 3: Number of pixel (value of background is near to 1).
Number of pixel
Figure 7
Figure 9
exclude the value Type1 Type2 Type1 Type2of background
2930
2129
298250Figure 10: Four areas of output images.

4. Conclusions
the background value is large, it is easier to extract the moving object. We calculate the number of pixel in whole image
exclude the value of −0.8 ∼ −0.2 (background value) in Figs.
6 and 8. Table 2 shows the calculated results. For the case ofFig. 6, the number of pixel is large by using “Type1” whichmeans that “Type1” equation is better to extract the moving
object when background color is more dark than the object.
On the other hand, in the case of Fig. 8, moving object is more
extracted by using “Type2” when background color is morebright than the object.

In this research, we have investigated the characteristics ofoutput values in D-CNN for motion pictures. We compared
the output values of the cell by two update equations. Fromsimulation results, we can see that when the moving object ismore bright than the background, “Type1” equation performs
better than “Type2” for the extracting moving object. On theother hand, when the moving object is more dark than thebackground, “Type2” equation performs well.
References

Table 2: Number of pixel (value of background is near to -1). [1] L. O. Chua and L. Yang, “Cellular Neural Networks:Theory,” IEEE Trans. Circuits Syst., vol. 32,
Number of pixel
Figure 6
Figure 8
pp. 1257-1272, Oct. 1988.
exclude the value Type1 Type2 Type1 Type2of background73921814012075[2] M. Kawahara, T. Inoue and Y. Nishio, “Cellular Neural
Network with Dynamic Template and its Output CharacAlso, we calculate the number of pixel in whole images
teristics,” Proc. of IJCNN’09, pp. 1552-1558, Jun. 2009
exclude the value of 0.2 ∼ 0.8 (background color) in Figs. 7
and 9. Table 3 shows the calculated results. For the case of [3] M. Kawahara, T. Inoue and Y. Nishio, “Image Processing
Application Using CNN with Dynamic Template” Proc.
Fig. 7, the number of pixel is large by using “Type1” whichof CNNA’10, pp. 41-46, Feb. 2010.
means that “Type1” equation is better to extract the movingobject when background color is more dark than the object. [4] M. Kawahara, Y. Uwate and Y. Nishio, “CharacterisOn the other hand, in the case of Fig. 9, the number of pixel oftics of Cellular Neural Networks with Dynamic Tem“Type1” and “Type2” has similar values. This is because, forplate for Motion Pictures” Proc. of NOLTA’10, pp. 23-26,
the input image Fig. 5, the background and the object color
Sep. 2010.
are not so diﬀerence. So, there are not so much diﬀerence- 63 -