IEICE TRANS. FUNDAMENTALS, VOL.E85–A, NO.9 SEPTEMBER 2002

2052

PAPER

Special Section on Nonlinear Theory and Its Applications

Image Processing of Two-Layer CNNs—–Applications andTheir Stability—–
Zonghuang YANG†a) , Student Member, Yoshifumi NISHIO†† ,and Akio USHIDA†† , Regular Members

SUMMARY Cellular Neural Networks (CNNs) have beendeveloped as a high-speed parallel signal-processing platform. In
this paper, a generalized two-layer cellular neural network modelis proposed for image processing, in which two templates are
introduced between the two layers. We found from the simulations that the two-layer CNNs eﬃciently behave compared to thesingle-layer CNNs for the many applications of image processing.
For examples, simulation problems such as linearly non-separable
task—logic XOR, center point detection and object separation,etc. can be eﬃciently solved with the two-layer CNNs. The stability problems of the two-layer CNNs with symmetric and/or
special coupling templates are also discussed based on the Lyapunov function technique. Its equilibrium points are found fromthe trajectories in a phase plane, whose results agree with thosefrom simulations.
key words: cellular neural networks, image processing, cloning
template, stability

1.

Introduction

Since Cellular Neural Networks (CNNs) were proposedby Chua and Yang [1], [2], they have been successfully
developed for various high-speed parallel signal processings such as image processing [3]–[10] as well as modeling of nonlinear phenomena [16]–[21].
Up to the present, many kinds of templates for
single-layer and two-layer CNNs have been already proposed for special purposes. Reference [15] provides
many templates and algorithms for the applications ofimage processing with single-layer CNNs. The image
processing such as center point detection, skeletonizing and object separation etc. with single-layer CNNs,
can be carried out by the iterative use of diﬀerent
time-invariant templates [11]–[14], where each singlelayer CNN is iteratively used to perform a part of thetask. After the operation of CNN has attained to thesteady state or reached at some state, the next singlelayer CNN begins to perform the next part of the task.
In this way, the process iteratively continues until thewhole task is completed. This procedure is really teManuscript received November 5, 2001.
Manuscript revised March 25, 2002.
Final manuscript received May 24, 2002.
†
The author is with the Graduate School of Engineering,
University of Tokushima, Tokushima-shi, 770-8506 Japan.
††
The authors are with the Department of Electrical and Electronic Engineering, Universuty of Tokushima,
Tokushima-shi, 770-8506 Japan.
a) E-mail: yangzh@ee.tokushima-u.ac.jp

dious serial image processing. On the other hand, it iswell known in the neural networks that if a multi-layer
structure neural network is taken into consideration,
it will have more wide applications in diﬀerent ﬁelds
such as linearly non-separable problem, optimizationand pattern recognition, etc. [32]. This mechanism willbe also valid to the cellular neural networks. Actually, the concept and the interesting applications of thetwo and more layer CNN structures have already been
proposed in [1], [16]–[21], [27]–[29], where the CNNs areused in generating nonlinear phenomena such as autowaves, pattern formation, Radon transform of a binary image and so on. Another interesting applicationof two-layer CNN in [30] was proposed as highly parallel
method for mapping and navigation of an autonomous
robot.
In this paper, we will systematically investigate theability of CNNs with two-layer structure in image processing. Through our many experimental examples, wefound that the two-layer CNNs eﬃciently behave compared to the single-layer CNNs for many image processing applications.
This paper is organized as follows: In Sect. 2, a
two-layer cellular neural network model is described as
the extension of the single-layer CNN [1], in which two
coupling templates are introduced. In Sect. 3, several
interesting simulations for image processing are given
in detail. All of them show that the two-layer CNNs are
more eﬃcient for the applications of image processing
compared to single-layer CNNs. In Sect. 4, the stability of the two-layer CNN with symmetric and/or special
coupling templates is discussed based on the Lyapunov
function technique, whose equilibrium points are calculated by the trajectories in the phase plane.
2.

Two-Layer CNN Architecture

Now, let us formulate the system equations of the twolayer CNNs by introducing two coupling templates C1and C2 . We assume that each layer of the CNN is composed of a two-dimensional M by N array structure asshown in the left hand side of Fig. 1. Each cell in thearray is denoted by c(i, j), and has two state variables
x1 (i, j), x2 (i, j), where (i, j) stands for the position ofa cell in the array, for 1 ≤ i ≤ M and 1 ≤ j ≤ N .
The state equations of each cell are given by two ﬁrst-

YANG et al.: IMAGE PROCESSING OF TWO-LAYER CNNS—APPLICATIONS AND THEIR STABILITY—

2053

f (x1,ij )

u1
B1

First
layer

N

f (x2,ij )
1

A1

C1

-2

C2

Fig. 2 The CNN output function described by thepiecewise-linear sigmoid characteristic.

Secondlayer u2 B2
A cell c(i, j)
M
Fig. 1 A two-dimensional cellular neural network, andcoupling between the ﬁrst and second layers.

u1order diﬀerential equations given by Eq. (1), and theoutput equations are given by Eq. (2), where f (·) is apiecewise-linear nonlinear function deﬁned by Eq. (3).
Figure 2 shows the characteristics of the output function. We deﬁne the state variables of the ﬁrst layer by
x1 (i, j), and those of the second layer by x2 (i, j). u
and y refer to the input and output variables of thecell. A(i, j; k, l), B(i, j; k, l), and I mean the feedback
template, ﬁlter template and bias current, respectively.
The index 1 and 2 stand for the ﬁrst layer and second
layer of the two-layer CNN array. C1 (i, j; k, l) is thecoupling template to transfer the second layer output
to the ﬁrst layer input, and C2 (i, j; k, l) is vice versa.
They also show the weight of the local couplings among
the cells in the neighborhood Nr (i, j) as the templatesA(i, j; k, l) and B(i, j; k, l). r is the coupling radius. In
general, r takes the value of 1 or 2. An example ofr = 1 for a particular cell is shown by the gray cells in
Fig. 1.
dx1,ij= −x1,ij + I1



dt


+ C(k,l)∈Nr (i,j) A1 (i, j; k, l)y1,kl 





+ C(k,l)∈Nr (i,j) B1 (i, j; k, l)u1,kl + C(k,l)∈Nr (i,j) C1 (i, j; k, l)y2,kl (1)dx2,ij= −x2,ij + I2


dt



+ C(k,l)∈Nr (i,j) A2 (i, j; k, l)y2,kl + C(k,l)∈Nr (i,j) B2 (i, j; k, l)u2,kl + C(k,l)∈Nr (i,j) C2 (i, j; k, l)y1,kl


f (x) = 0.5(|x + 1| − |x − 1|).

2

-1

A2

y1,ij = f (x1,ij )y2,ij = f (x2,ij )

1

0

-1

x1,ij
x2,ij

(2)
(3)

The block diagram of two-layer CNN is shown by
Fig. 3, which constitutes a closed-loop system. Observethat they share out the tasks in image processing andcooperate with each other, which cannot be performed
with single-layer CNNs. Note that CNNs proposed insome literatures [11]–[15] are serial image processing
with single-layer CNNs, and our two-layer CNNs on

First layer

−1

B1
ẋ1



y1

f (·)
x1

I1

A1
C2

C1
A2
f (·)

x2



I2
ẋ2

y2
Second layer
Fig. 3−1

B2

u2

Block diagram of the two layer CNN.

the contrary are parallel image processing. Moreover,when one of the templates C1 and C2 is set to zero, theCNN becomes an open-loop system and behaves as thecascade connection of two single-layer CNNs. Therefore, the two-layer CNNs will have wide applications inimage processing compared to the single-layer CNNs.
3.

Applications in Image Processing

Image processing is one of the most important applications of the CNNs, and there have been alreadyproposed many kinds of templates for the single-layer
CNNs. In this section, we will give some important
examples of the two-layer CNNs.
3.1 Linearly Non-separable Problems
In binary value image processing, some logic operations between two images are often introduced suchas Logic OR, AND and NOT, etc. The Logic OR,
AND and NOT belong to linearly separable problems
in image processing, whose templates with single-layer
CNNs have been reported in the literature [15]. In fact,
this kind of CNNs is extremely useful to realize theselogic operations. For some linearly non-separable tasks(e.g. Logic XOR), they cannot be directly solved by thesingle-layer CNNs. However, we can solve them withthe two-layer CNNs, directly.
Now, let us consider a simpliﬁed two-layer CNN,
i.e., all parameters besides the center positions in the

IEICE TRANS. FUNDAMENTALS, VOL.E85–A, NO.9 SEPTEMBER 2002

2054

dx1 /dt
dx2 /dt
1
-2

-1

1

0

2

x1,ij
x2,ij

-1
g1 and g2

Fig. 4 The trajectories of state variables for various g1 andg2 , where we set a1,00 = a2,00 = 2 and the stable and unstable equilibrium points are denoted by the solid dots and circles,
respectively.

templates A1 , A2 , B1 , B2 , C1 and C2 are zero. For
simpliﬁcation, we can omit the subscript (i,j), because
each cell in the CNNs is coupled in the same way. Thus,
the cell equations are written as follows:ẋ1 = −x1 + a1,00 y1 + b1,00 u1 + c1,00 y2 + I1 
= −x1 + a1,00 y1 + g1
(4)ẋ2 = −x2 + a2,00 y2 + b2,00 u2 + c2,00 y1 + I2 = −x + a y + g
22,00 22

For the self-feedback coeﬃcients a1,00 > 1 and a2,00 >1, the trajectories of the state variables without g1 andg2 behave like as the solid line in Fig. 4. The trajectories
of Eq. (4) can be obtained by shifting g1 and g2 values
up and down as shown by the dotted lines in the ﬁgure.
Observe that if we choose the initial conditions x1 (0)and x2 (0) larger than the unstable equilibrium points
shown by circles, the steady-state output will be given
by 1, and otherwise, the outputs will be given by −1.
Thus, we have the following output relations dependingon the initial conditions;


g1



y1 = sgn x1 (0) +
a1,00 − 1b1,00 u1 + c1,00 y2 + I1 = sgn x1 (0) +
a1,00 − 1

(5)

g2


y2 = sgn x2 (0) +
a2,00 − 1b2,00 u2 + c2,00 y1 + I2 = sgn x2 (0) +a2,00 − 1
where b1,00 , b2,00 , c1,00 , c2,00 , I1 and I2 are the constantparameters. u1 and u2 are the inputs of cells, which areusually equal to 1 or −1 corresponding to black or white
pixel in the binary images, or are between 1 and −1 forthe gray scale images. Therefore, by suitably selecting
these parameters and initial state conditions, we canobtain the expected outputs.
For example, consider the two-layer CNN to perform a linearly non-separable task—logic XOR. The
logic function can be written as

Fig. 5 An example for solving a linearly non-separable problem. (a), (b) and (c) are the input, the initial state condition
and the output of the ﬁrst layer, (d), (e) and (f) are the input,
the initial state condition and the output of the second layer,
respectively.

F = X1 ⊕ X2 .

(6)

In this logic, the input of two pixels has four possiblecombinations, and the output will be black only if oneof the two pixels is black. The templates are:




0 0 0
0 0 0
A1=A2= 0 2 0  , B1=B2=  0 1 0  ,
0 0 0
0 0 0


0 0 0
C2= 0 −2 0  , C1 = 0, I1 = −1, I2 = −1.
0 0 0
(7)
Consider an example for solving a linearly nonseparable problem shown in Fig. 5. One of the two given
images is set to the input u1 of the ﬁrst layer CNN and
the initial state x2 (0) of the second layer. Another image is set to the input u2 of the second layer and thestate x1 (0) of the ﬁrst layer. The execution result isobtained from the output of the second layer as shownby Fig. 5, where (a) and (d) are two input images of
the two layers, and (b) and (e) are the initial states.
The execution result is shown by (f), where the black
pixels are obtained only when one of the two inputs has
a black pixel, and white pixels are obtained for every
other case. The output of the ﬁrst layer shows the result obtained by Logic AND of the two images. These
results can be calculated by the relations Eq. (5). From
the above simulation result, we have shown that thesimpliﬁed two-layer CNNs are capable to perform linearly non-separable tasks.
3.2 Compound Tasks
There are many complicated tasks such that we need toapply some steps of single-layer CNN operations suchas Logic operation, edge detection and diﬀusion, etc.

YANG et al.: IMAGE PROCESSING OF TWO-LAYER CNNS—APPLICATIONS AND THEIR STABILITY—

2055

Fig. 6 Endocardial boundary detection (a) the initial statesand inputs of both two layers, (b) the output of the ﬁrst layer,(c)the output of the second layer.

in the step by step manner. Hence, these processes are
tedious and complicated to choose their templates. We
show in this section that it can be directly executed by
the one step of a two-layer CNN.

Fig. 7 An example for solving object separation problem. (a),(b) and (c) are the initial state condition, the transient result andthe output of the ﬁrst layer. (d) is the input of the ﬁrst layer andthe initial state condition of the second layer, (e) and (f) are thetransient results and the output of the second layer, respectively.

3.2.1 Feature Detection of Endocardial Image
Medical image processing is one of major topics ofCNNs [31]. The boundary detection, which is one of
the feature extraction techniques, is of great importance both for qualitative and quantitative analysis forsome medical image processing. Its image will help thedoctors to correctly examine the patient’s conditions.
In this subsection, let us consider such an example thata two-layer CNN will detect the endocardial boundaryof left ventricle from ultrasonic (US) image as shownin Fig. 6(a). Assume that we want to detect the endocardial boundary and superimpose it on the original
image. This process can be more eﬃciently executedby the two-layer CNN in one step, compared to thesingle-layer CNNs [31].
Namely, the ﬁrst layer CNN executes a thresholdprocess with certainly diﬀusing and ﬁltering. The second layer detects the edges from the ﬁrst layer output
with C2 template and superimposes them on the original image. The ﬁnal result can be obtained in thesecond layer output. The templates are given as follows:
0 1.6 00 0.5 0A1= 1.6 2 1.6  , B1=0, C1= 0.5 2 0.5  ,0 1.6 00 0.5 0




0 0 0
−1 −1 −1
A2= 0 −1 0  , B2=2, C2= −1 8 −1  ,0 0 0−1 −1 −1
I1 = 0, I2 = −1.

(8)

In this case, the original US image is set as both twolayers as the initial states, and inputted to second layer
as shown in Fig. 6(a). The input of the ﬁrst layer may
be arbitrary due to B1 = 0. In the simulation, we set
the ﬁrst layer input as the same US image and appliedthe zero-ﬁxed boundary condition to both two layers.

Thus, we obtained the expected result Fig. 6(c) by superimposing the endocardial boundary on the original
image. Figure 6(b) is binary endocardial image separated from the original image.
3.2.2

Object Separation

In this subsection, we consider another application tocontinuously extract a character from a word, and recognize it. After a character is extracted, it must be
erased from the original text. This process can be recognized as an object separation. Now, let us consider asimple case separating the round and triangle objects
from the three images of the round, rectangular and triangle as shown in Fig. 7(d). This task can be executedby two processes such as extracting the chosen objects
by setting two pointers in advance as the initial statesat the positions of the round and triangle objects inthe ﬁrst layer, and erasing the extracted objects fromthe original image at the same time in the second layer.
This task can be similarly carried out by a single stepof two-layer CNN as follows.
The original binary image shown by Fig. 7(d) is inputted to the ﬁrst layer input and set as the initial stateof the second layer. By setting two pointers in advanceat the ﬁrst layer as the initial state shown by Fig. 7(a),we can extract the speciﬁed objects under the consideration with recall template [15]. Its output is transferredto the second layer through C2 . Then, we can executethe logic diﬀerence at the second layer with logic difference template [15]. Thus, the extracted two objects
are obtained by the ﬁrst layer output, and the rest rectangle object is obtained by the second layer output asshown by Figs. 7(c) and (f), respectively. Figures 7(b)and (e) are the transient results of the ﬁrst and second
layers. In this example, we also have chosen zero-ﬁxed

IEICE TRANS. FUNDAMENTALS, VOL.E85–A, NO.9 SEPTEMBER 2002

2056

boundary condition [1]. The templates are
0.5 0.5 0.5
0 0 0
A1 =  0.5 2 0.5  , B1 =  0 4 0  , C1 = 0,
0.5 0.5 0.5
0 0 0




0 0 0
0 0 0
A2 =  0 2 0  , B2 = 0, C2 =  0 −1 0  ,
0 0 0
0 0 0
I1 = 0.25, I2 = −1.

Fig. 8

Table 1 The landscape and stable equilibrium point of a cellin the ﬁrst layer.

(9)

From the above two examples, we can conclude
that the two-layer CNN can execute each compoundtask in one step, where each layer shares out the tasks
and cooperates in each other. However, if we use singlelayer CNNs, it will take two or more steps in the sequential manner. Thus, the two-layer CNN is more eﬃcientcompared to the single-layer CNNs.
3.3 Complex Tasks
In many applications of image processing and patternrecognition, it is very important to detect the center
point of a given object, because it is used as the reference position of the object. Unfortunately, the definition of the center point is very ambiguous, because
there are various objects such as convex, concave, disk
with white holes, etc. In this paper, we deﬁne “thecenter point is located at the halfway from the furthermost points of a given object image.” Therefore, atﬁrst, a given object image is changed into a rectangular block including a given object with shadow template[15]. Thus, the center point problem is reduced to the
problem of ﬁnding out the middle point of the rectangular object. If a single-layer CNN is applied to the center
point problem, we need to progressively peel oﬀ one column or row pixels from four directions—left, bottom,
right and up hand sides. These sequential processes
are continued to get the center point. The number of
iterations depends on the size of the object. This is avery complicated process. If we use a two-layer CNN,
this task can be solved by two steps. The ﬁrst step isto detect the center line of the object, and the second
step is to detect the center point of the above centerline.
3.3.1 Detection of the Center Line of Object
Now, consider a center line detection problem shown in
Fig. 9(a). The object image is set to the initial statesof both CNN layers as shown by Figs. 9(a) and (g). We
used the following templates to simultaneously peel oﬀ
the most left and right hand side pixels:




0 0 0
0 0 0
B1 = 0,A1= 0.5 2 0.5  , C1= 0 2.8 −2  ,I1 = −2,
0 0 0
0 0 0

All possible landscapes of a cell.

Landscapecase (a)case (b)case (c)case (d)case (e)case (f)case (g)

g1,ij
−3.8
−6.8
−1.2
−0.2
2.8
1.8
−2.8

initial state
−1
−1
1
1
1
1
−1

stable output
−1
−1
−1
1
1
1
−1






0 0 0
0 0 0
B = 0,
A2= 0.5 2 0.5  , C2= −2 2.8 0  , 2
I2 = −2,
0 0 0
0 0 0

(10)

The cell’s state equations can be rewritten in the following forms:
ẋ1;ij = −f1 (x1;i,j ) + g1
(11)ẋ2;ij = −f2 (x2;i,j ) + g2
where
f1 (x1;i,j ) = x1;i,j − (|x1;i,j + 1| − |x1;i,j − 1|),f2 (x2;i,j ) = x2;i,j − (|x2;i,j + 1| − |x2;i,j − 1|),g1;i,j = 0.5y1;i,j−1 +0.5y1;i,j+1 +2.8y2;i,j −2y2;i,j+1 −2,
g2;i,j = 0.5y2;i,j+1 +0.5y2;i,j−1 −2y1;i,j−1 +2.8y1;i,j −2.
(12)
To analyze the stability of the equilibrium point of acell, we restrict the discussion to a landscape of the cellby listing all those neighboring states, including itself.
Since the two state x1 (0) and x2 (0) are the same at
the beginning of this simulation, the landscape can bedivided into seven possible cases as shown in Fig. 8, in
which the black squares denote cells having value 1 andthe white squares −1 valued cells. On the other hand,the crossed square cells stand for ‘don’t care’ for thecenter cell, because the template coeﬃcients in Eq. (10)corresponding to those positions are zeros.
For the case of Fig. 8(a), we have
g1;i,j = 0.5y1;i,j−1 +0.5y1;i,j+1 + 2.8y2;i,j − 2y2;i,j+1 −2= 0.5×(−1)+0.5×(−1)+2.8×(−1)−2× (−1)−2= −3.8.
In this case, we have y1 = sgn[x1 (0) − 3.8] from the relations (5). Thus, we have y1 = −1 (white) for x1 (0) ≤−1. Similarly, for the other cases from Figs. 8(b) to (g),their stable equilibrium points can be obtained as listed
in Table 1. Observe from the table that the output of
the center cell only in the case (c) will be changed from

YANG et al.: IMAGE PROCESSING OF TWO-LAYER CNNS—APPLICATIONS AND THEIR STABILITY—

2057

Fig. 9 An example for extracting the center line of an object. States from (a) to (f) are
the output of the ﬁrst layer in the time progress, and those from (g) to (l) are the output
of second layer. Final results of the two layer CNN are shown by (f) and (l).

1 (black) to −1 (white), and unchanged for the othercases. Thus, the ﬁrst layer of the CNN performs apeeling-oﬀ of the leftmost pixels of the object as shownin Fig. 9(b). By the similar analysis, the second layer
performs a peeling-oﬀ of the rightmost pixels of the object as shown in Fig. 9(h). These peelings are executedat the same time in each layer. Thus, we get the middle results as shown by Figs. 9(c) and (i), respectively.
After that, the landscape can be divided into eight possible cases as shown in Fig. 10. At this moment, we have
g1;i,j = −2.8 and y1 = sgn[x1 (0) − 2.8] = −1 for caseFig. 10(g), g2;i,j = −2.8 and y2 = sgn[x2 (0) − 2.8] = −1
for case Fig. 10(c). Thus, both the rightmost pixels of
the ﬁrst layer and the leftmost pixels of the second layer
are peeled oﬀ as shown in Figs. 9(d) and (j), respectively. Figures 9(e) and (k) are the same as (a) and(g), except for the both end pixels which are peeledoﬀ. Thus, one cycle of the peeling-oﬀ is completed.
This cycle continues until the vertical centerline is remained, where the transient has arrived at the steadystate corresponding to Fig. 9(f). This dynamic process
can be observed from the graph of the sum of Stateerror Square varying with the integration time shownin Fig. 11. The sum of State-error Square is deﬁned as
SCN N (t) =

M 
N


Fig. 10

All possible landscapes of a cell at a transient.

((x1,ij (t + ∆t) − x1,ij (t))2i=1 j=1+ (x2,ij (t + ∆t) − x2,ij (t))2 )(13)

The peaks show the states when the pixels from the twolayers are unbalanced states. When the object is completely peeled into center line, the sum of state-errors
square decays to zero. The two layer stable outputs areshown by Figs. 9(f) and (l), respectively. In this simulation, the zero-ﬁxed boundary condition is also adopted.

Fig. 11 The sum of State-error Square via time for center line
extraction.

3.3.2

Detection of the Center Point from the AboveLine

To detect the center point from the above center line,
we need to apply the same algorithm to the center linewith the transposed templates as follows:IEICE TRANS. FUNDAMENTALS, VOL.E85–A, NO.9 SEPTEMBER 2002

2058

−

1
C1 (i, j; k, l)y1,ij y2,kl
2(i,j) (k,l)

+


1 2
y1,ij −
I1 y1,ij
2
(i,j)

−

1

(i,j)

A2 (i, j; k, l)y2,ij y2,kl
2(i,j) (k,l)
+
B2 (i, j; k, l)y2,ij u2,kl(i,j) (k,l)−

1
C2 (i, j; k, l)y2,ij y1,kl
2(i,j) (k,l)


1 2
+
y2,ij −
I2 y1,ij
2Fig. 12 An example for extracting the center point of line. (a),(b) and (c) are the initial state condition, the transient result
and the output of ﬁrst layer. (d), (e) and (f) are the initial state
condition, the transient result and the output of the second layer,
respectively.

(i,j)

Therefore, we can show that the function E(t) isbounded as follows:
max |E(t)| ≤ Emax
t






0 0.5 00 0 0
B1 = 0,A1= 0 2 0  , C1= 0 2.8 0  ,I1 = −2,
0 0.5 00 −2 0
0 0.5 00 −2 0
B = 0,
A2= 0 2 0  , C2= 0 2.8 0  , 2
I2 = −2,
0 0.5 0
0 0 0

Where

(14)

Since the CNNs are usually required to be stable for theimage processing applications, the studies on complete
stability have been vigorously discussed and many criteria have been obtained [1], [23]–[26]. In this section,
we will discuss the convergence property, and its related problems for the two-layer CNNs with symmetricand/or special coupling templates.
For analyzing the convergence properties of dynamic nonlinear systems, one of the most eﬀective techniques is Lyapunov’s method, which has been successfully applied to the stability analysis of single-layer
CNN [1]. In this section, we analyze the stability ofthe two-layer CNN with the same techniques. We deﬁne a Lyapunov function E(t) of a two-layer CNN by
the scalar function, which is similar to the one usedin [1], [22] and can be interpreted as the “generalizedenergy” for a two-layer CNN.
1
A1 (i, j; k, l)y1,ij y1,kl
2(i,j) (k,l)
+
B1 (i, j; k, l)y1,ij u1,kl(i,j) (k,l)

1
|A1 (i, j; k, l)|2(i,j) (k,l)
+|B1 (i, j; k, l)|(i,j) (k,l)+

1
|C1 (i, j; k, l)|2(i,j) (k,l)

1
+|A2 (i, j; k, l)|2(i,j) (k,l)
|B2 (i, j; k, l)|+Stability of Two-Layer Cellular Neural Networks

E(t) = −(16)

Emax = +The transient behaviors are shown in Fig. 12. Thus, we
can ﬁnd the center point in two steps.
4.

(15)

(i,j)

(i,j) (k,l)+

1
|C2 (i, j; k, l)|2(i,j) (k,l)+ M N (1 + |I1 | + |I2 |)(17)

Now, let us assume the templates satisfy the following
condition Eq. (18):A1 (i, j; k, l) = A1 (k, l; i, j)

A2 (i, j; k, l) = A2 (k, l; i, j)(18)
C1 (i, j; k, l) = C2 (k, l; i, j) C (i, j; k, l) = C (k, l; i, j)
2

1

From Eq. (15), we have the following relation:  dy1,ij 2  dy2,ij 2
dE(t)
=−
+
≤ 0 (19)
dt
dt
dt
(i,j)

Thus, we found that the energy function is monotone
decreasing, where we used the following constraint conditions:dy1,ij1 |x1,ij | < 1=(20)
0 |x1,ij | ≥ 1
dx1,ijYANG et al.: IMAGE PROCESSING OF TWO-LAYER CNNS—APPLICATIONS AND THEIR STABILITY—

2059

x1,ij = y1,ij , |x1,ij | < 1
dy2,ij=dx2,ij



(21)
5.

1
0

|x2,ij | < 1|x2,ij | ≥ 1

x2,ij = y2,ij , |x2,ij | < 1

(22)

(23)

We found that from Eqs. (16) and (19), for any given
inputs u1 , u2 and the initial states x1 , x2 , we obtainthat
lim E(t) = const.

(24)

dE(t)
=0
dt

(25)

t→∞

and
lim

t→∞

Thus, under the condition Eq. (18), we always have thestable steady state outputs in both two layers after thetransient. Now, we show that both the states x1 and x2
approaches to the equilibrium points. Let us consider
the system equations Eqs. (1)–(3) again, and rewrite
them in the following form:
dx1,ij (t)= −f1 (x1,ij (t)) + g1 (t)dt
(26)
dx2,ij (t)= −f2 (x2,ij (t)) + g2 (t)
dt
where
1
f1(x1,ij ) = x1,ij − A1 (i, j; i, j)(|x1,ij +1|−|x1,ij −1|)21
f2(x2,ij ) = x2,ij − A2 (i, j; i, j)(|x2,ij +1|−|x2,ij −1|)2g1 (t) =A1 (i, j; k, l)y1,kl + C1 (i, j; k, l)y2,kl(k,l)=(i,j)



+

(k,l)

B1 (i, j; k, l)u1,kl + I1(k,l)

g2 (t) =
A2 (i, j; k, l)y2,kl +C2 (i, j; k, l)y1,kl(k,l)=(i,j)

+



(k,l)

B2 (i, j; k, l)u2,kl + I2

(k,l)

Observe that g1 (t) and g2 (t) are not related to thestates x1,ij and x2,ij , and are only the functions of
the outputs (y1 , y2 ), inputs (u1 , u2 ) and biases (I1 , I2 ).
Therefore, the trajectories of Eq. (26) for A1 (i, j; i, j) >1 and A2 (i, j; i, j) > 1 have the same structures asshown in Fig. 4, and have the stable and unstable equilibrium points. Thus, we can conclude for the condition Eq. (18) that the steady state of two-layer CNN
is completely stable, in the meaning that both statesand outputs are in the steady state. The validity of theabove analysis has been veriﬁed by the above simulations shown in Sects. 3.3.1 and 3.3.2.

Conclusions

In this paper, we have proposed a two-layer cellularneural network model, and discussed the applications
and stability. We found that it has many interesting
applications such as linearly non-separable task—logic
XOR, some compound tasks and complex problems—
the center point detection of the object. Although, ofcourse, they can be solved by single-layer CNNs, theapplications of the two-layer CNNs to these problems
are very eﬃcient compared to the single-layer CNNs.
Especially, the center point detection problem with thetwo-layer CNNs can be found the solution in two steps,
however, if we use the single-layer CNNs, it will befound many steps depending on the object size in serial
manner. We can prove the stability for the special casesof two-layer CNNs, where its necessary conditions areproved by the Lyapunov function and behaviors of the
trajectories on the phase plane. In future problems, wewant to ﬁnd the special applications of the two-layer
CNNs such that they cannot be solved by the singlelayer CNNs.
References[1] L.O. Chua and L. Yang, “Cellular neural networks: Theory,” IEEE Trans. Circuits & Syst., vol.35, no.10, pp.1257–
1272, Oct. 1988.
[2] L.O. Chua and L. Yang, “Cellular neural networks: Applications,” IEEE Trans. Circuits & Syst., vol.35, no.10,
pp.1273–1290, Oct. 1988.
[3] M. Ikegami and M. Tanaka, “Image coding and decoding
by discrete time cellular neural networks and its error evaluation,” IEICE Trans. Fundamentals (Japanese Edition),
vol.J77-A, no.7, pp.954–964, July 1994.
[4] M. Tanaka, K.R. Crounse, and T. Roska, “Parallel analog image coding and decoding by using cellular neuralnetworks,” IEICE Trans. Fundamentals, vol.E77-A, no.7,
pp.1387–1395, July 1994.
[5] M. Ohnishi, M. Ikegami, Y. Tanji, K. Jinno, and M. Tanaka,“Image intensity conversion using discrete time cellular neural networks,” J. Signal Processing, vol.3, no.6, pp.491–498,
Nov. 1999.
[6] C. Rekeczky, A. Ushida, and T. Roska, “Rotation invariant
detection of moving and standing objects using analogic
cellular neural network algorithms based on ring-codes,”IEICE Trans. Fundamentals, vol.E78-A, no.10, pp.1316–
1330, Oct. 1995.
[7] T. Roska and L.O. Chua, “The CNN universal machine,”
IEEE Trans. Circuits & Syst., vol.40, no.3, pp.163–173,
March 1993.
[8] L.O. Chua and T. Roska, “The CNN pradigm,” IEEE
Trans. Circuits & Syst., vol.40, no.3, pp.147–156, March
1993.
[9] L.O. Chua and T. Roska, “The CNN universal machine: An
analogic array computer,” IEEE Trans. Circuits & SystemII, vol.40, no.3, pp.163–173, March 1993.
[10] F. Werblin, T. Roska, and L.O. Chua, “The analogic cellular neural network as bionic eye,” Int. J. Circuit Theory &Appl., vol.23, pp.541–569, 1995.
[11] P.L. Ventianer, F. Werblin, T. Roska, and L.O. Chua,“Analogic CNN algorithms for some image compression

IEICE TRANS. FUNDAMENTALS, VOL.E85–A, NO.9 SEPTEMBER 2002

2060

and restoratin tasks,” IEEE Trans. Circuits & Syst., vol.42,
no.5, pp.278–284, May 1995.
[12] H. Harrer and J.A. Nossek, “Skeletonization: A new application for discrete-time cellular neural networks using
time-variant templates,” Proc. IEEE Int. Symp. Circuits& Syst., pp.2897–2900, 1992.
[13] D. Yu, C. Ho, X. Yu, and S. Mori, “On the applicationof cellular automata to image thinning with cellular neuralnetwork,” Proc. IEEE Int. Wkshp. Cellular Neural Networks and Their Applicat, CNNA-92, pp.210–215, 1992.
[14] T. Matsumoto, T. Yokohama, H. Suzuki, R. Furukawa, A.
Oshimoto, T. Shimmi, Y. Matsushida, T. Seo, and L.O.
Chua, “Several image processing examples by CNN,” Proc.
IEEE Int. Wkshp. Cellular Neural Networks and Their Applicat., CNNA-90, pp.100–112, 1990.
[15] T. Roska, L. Kek, L. Nemes, A. Zarandy, and P. Szolgay,“CNN software library (Templates and algorithms),” Analogical and neural computing laboratory, Computer andAutomation Institute of the Hungarian Academy of Sciences, Version 7.3, 1999.
[16] L.O. Chua, M. Hasler, G.S. Mochytz, and J. Neirynck,“Autonomous cellular neural networks: A uniﬁed paradigm
for pattern formation and active wave propagation,” IEEE
Trans. Circuits & Syst., vol.42, no.10, pp.559–577, 1995.
[17] Z.H. Yang, Y. Nishio, and A. Ushida, “A model of nonlinear phenomena: A uniﬁed framework two layer CNN,”
Proc. the 2001 international technical conference on circuits/systems, computer and communications, pp.1288–
1291, 2001.
[18] L. Goras, L.O. Chua, D.M.W. Leenaerts, and L. Pivka,“Turing patterns in CNNs part I–III,” IEEE Trans. Circuits& Syst., vol.42, no.10, pp.602–637, Oct. 1995.
[19] M.J. Ogorzalek, Z. Galias, A.M. Dabrowski, and W.R.
Dabrowki, “Chaotic waves and spatio-temporal patterns
in large arrays of doubly-coupled Chua’s circuits,” IEEE
Trans. Circuits & Syst., vol.42, no.10, pp.706–714, 1995.
[20] L. Pivka, “Autowave and spatio-temporal chaos in CNNs
Part I and II: A tutorial,” IEEE Trans. Circuits & Syst.,
vol.42, no.10, pp.638–664, 1995.
[21] P. Thiran, K.R. Crounse, L.O. Chua, and M. Hasler, “Pattern formation properties of autonomous cellular neuralnetworks,” IEEE Trans. Circuits & Syst., vol.42, no.10,
pp.757–774, 1995.
[22] J.J. Hopﬁeld, “Neurons with graded response have collective computational properties like those of two-state neurons,” Proc. Natl. Acad. Sci. USA, vol.81, pp.3088–3092,
1984.
[23] C.W. Wu and L.O. Chua, “More rigorous proof of complete
stability of cellular neural networks,” IEEE Trans. Circuits& Syst. I, vol.44, no.4, pp.370–371, 1997.
[24] L.O. Chua and T. Roska, “Stability of a class of nonreciprocal cellular neural networks,” IEEE Trans. Circuits & Syst.,
vol.37, no.12, pp.1520–1527, 1990.
[25] L.O. Chua and C.W. Wu, “On the universe of stable cellularneural networks,” Int. J. Circuit Theory & Appl., vol.20,
pp.497–517, 1992.
[26] N. Takahashi and L.O. Chua, “On the complete stabilityof nonsymmetric cellular neural networks,” IEEE Trans.
Circuits & Syst. I, vol.45, no.7, pp.754–758, 1998.
[27] P. Arena, S. Baglio, L. Fortune, and G. Manganaro, “Complexity in a two-layer CNN,” Proc. IEEE Int. Wkshp.
Cellular Neural Networks and Their Applicat., CNNA-96,
pp.127–132, 1996.
[28] C.W. Wu, L.O. Chua, and T. Roska, “A two-layer radontransform cellular neural network,” IEEE Trans. Circuits& Syst. II: Analog and Digital Signal Processing, vol.39,
no.7, pp.488–489, 1992.

[29] C. Botoca, “An associative memory using a two layer cellular neural network, Part I,” Proc. International Conference
SCS 95, pp.121–124, Oct. 1995.
[30] B. Siemiatkowska, “A highly parallel method for mapping
and navigation of an autonomous mobile robot,” Proc.
International Conference on Robotics and Automation,
pp.2796–2801, San Diego, California, May 1994.
[31] C. Rekeczky, A. Tahy, Z. Vegh, and T. Roska, “CNN-based
spatio-temporal nonlinear ﬁltering and endocardial boundary detection in echocardiography,” Int. J. Circuit Theory& Appl., vol.27, pp.171–207, 1999.
[32] J.M. Zurada, Introduction to Artiﬁcial Neural Systems,
West Publishing Company, St. Paul, 1992

Zonghuang Yang
received the B.E.
degree in Radio Engineering from South
East University, Nanjing China in 1985,
and the M.E. degree in Electrical andElectronic Engineering from Tokushima
University, Tokushima Japan, in 2001. He
is currently working toward the Ph.D. degree at same department of Tokushima
University. His research interests include
theory and application of cellular neuralnetworks.

Yoshifumi Nishio
received the B.E.,
M.E., and Ph.D. degrees in electrical engineering from Keio University, Yokohama
Japan, in 1988, 1990, and 1993, respectively. In 1993, he joined the Department of Electrical and Electronic Engineering at Tokushima University, Tokushima Japan, where he is currently an
Associate Professor. From May 2000 hespent a year in the Laboratory of Nonlinear Systems (LANOS) at the Swiss Federal Institute of Technology Lausanne (EPFL) as a visiting professor. His research interests include analysis and application ofchaos in electrical circuits, analysis of synchronization in coupled oscillators circuits, development of analyzing methods fornonlinear circuits and theory and application of cellular neuralnetworks. Dr. Nishio is a member of the IEEE.

Akio Ushidareceived the B.E. and
M.E. degrees in Electrical Engineeringfrom Tokushima University in 1961 and
1966, respectively, and the Ph.D. degree
in Electrical Engineering from Universityof Osaka Prefecture in 1974. He was an
associate professor from 1973 to 1980 at
Tokushima University. since 1980, he has
been a professor in the Department ofElectrical and Electronic Engineering at
Tokushima University. From 1974 to 1975
he spent one year as a visiting scholar at the Department of Electrical Engineering and Computer sciences at the University ofCalifornia, Berkeley. His current research interests include numerical methods and computer-aided analysis of nonlinear systems. Dr. Ushida is a member of the IEEE.