Templates and Algorithms for Two-Layer
Cellular Neural Networks
Zonghuang Yang, Yoshifumi Nishio and Akio Ushida
Dept. of Electrical and Electronic Engineering,Tokushima University,
Minami-Josanjima 2-1, Tokushima 770-8506, JAPANyangzh@ee.tokushima-u.ac.jp

Abstract - This paper presents the two-layer cellular neural Networks for some important image processing applications, in which two new templates are
introduced to couple between the two layers. Several
simulations such as linear non-separable task, center
point detection and skeletonizing, are executed withthe two-layer CNN and their templates are given. Allof them display that the two-layer CNNs behave more
eﬃcient for image processing compared with singlelayer CNNs. In addition, the stability of the twolayer CNN with symmetric templates and/or special
coupling templates is also discussed based on the Lyapunov function technique. Its equilibrium points arefound from the trajectories in a phase plane. These
results agree with those from simulations.

I. INTRODUCTION
Owing to the eﬀorts of reseachers that have been donein Cellular Neural Networks (CNNs) [1], many kinds oftemplates have been already proposed for special purposes for single-layer CNNs. Many image processings,
such as center point detection, skeletonizing, etc., can be
carried out by the iterative use of diﬀerent time-variant
templates [2]-[4], where each single-layer CNN is iteratively used to perform a part of task. Namely, after
the operation of CNN has attained to the steady stateor reached at some state, the next single-layer CNN begins to perform the next part of the task. In this way,
the process iteratively continues until the whole task iscompleted. It is obvious that this procedure is tediousand belongs to serial processing. On the other hand, it
is well known in neural networks that if a multi-layer
structure neural network is taken into consideration, it
will have more widely applications in the diﬀerent ﬁelds
such as nonlinear separation, optimization and patternrecognition, etc. This mechanism will be also valid tocellular neural networks. In order to improve its applications, a cellular neural network model with two-layers
structure is introduced to solve some complicated image
processing problems in this paper. Throught our experimental examples, we observe that the two-layer CNN
structure behave more eﬃcient for image processing. Fi-

0-7803-7278-6/02/$10.00 ©2002 IEEE

nally, the stability of the two-layer CNN with symmetricand/or special templates is also discussed based on the
Lyapunov function technique; the equilibrium points aredemonstrated by the trajectries in the phase plane.
II. STRUCTURE OF TWO-LAYER CNN
The two-layer CNNs used in this paper is composed ofa two dimensional M by N array of cells. Each cell in
the array is denoted by c(i, j), and has two state variables
x1 (i, j), x2 (i, j), where (i, j) stands for the position of acell in the array, for 1 ≤ i ≤ M and 1 ≤ j ≤ N .
Now, we are going to formulate the system equationsof the two-layer CNNs by introducing two new counpling
templates C12 and C21 between two layers. The state
equations of each cell are given by two ﬁrst-order differential equations Eq.(1), and the output equations are
given by Eq.(2), where f (·) is a piecewise-linear nonlinear
function deﬁned by Eq.(3).
dx1,ijdt

dx2,ij
dt

=
+
+
+
=
+
+
+


−x


 1,ij + I1


A
(i,
j;
k,
l)y

1
1,kl

C(k,l)∈Nr (i,j)


B
(i,
j;
k,
l)u
1
1,kl 
C(k,l)∈N
(i,j)
r



C(k,l)∈Nr (i,j) C12 (i, j; k, l)y2,kl−x 2,ij + I2


A
(i,
j;
k,
l)y
2
2,kl 
C(k,l)∈N
(i,j)
r



B2 (i, j; k, l)u2,kl C(k,l)∈N
(i,j)

r


C(k,l)∈Nr (i,j) C21 (i, j; k, l)y1,kl
y1,ij = f (x1,ij )y2,ij = f (x2,ij )(1)

f (x) = 0.5(|x + 1| − |x − 1|).

(2)

(3)

We deﬁne state variables of the ﬁrst layer by x1 (i, j),and those of the second layer by x2 (i, j). u and y referto the input and output variables of the cell. A(i, j; k, l),B(i, j; k, l), and I are the feedback template, ﬁlter template and bias current, respectively. The index 1 and 2
stand for ﬁrst layer and second layer of the two-layer
CNN array. C12 (i, j; k, l) is the coupling template to

u1

−1
ẋ1 

B1

y1

f (·)

x1

I1

dx1 /dt
dx2 /dt

First layer

1

A1

-2
C21

C12

f (·)

x2

y2
Second layer

A2
 ẋ2

I2

−1

1

0

2

x1,ij
x2,ij

-1
g1 and g2

B2

u2

Fig. 1. Block diagram of the two-layer CNN

transfer the second layer output to the ﬁrst layer input,
and C21 (i, j; k, l) is vice versa. The two new templatesalso show the weight of the local coupling among the cells
in the neighborhood Nr (i, j) the same as A(i, j; k, l) andB(i, j; k, l). r is the coupling radius. In generally, r takes
the value of 1 or 2.
The block diagram of two-layer CNN is shown by Fig.1.
When both two C templates are zero, the CNN array
becomes obviously two independent single-layer CNNs.
When at least one of the templates C12 and C21 is not
zero, the CNN becomes an open-loop system with behaving as the cascade connections of two single-layer CNNs
or constitutes a closed-loop system. The two layers of
the CNN share out the tasks of some image processings and cooperate with each other, which cannot be
performed with single-layer CNN. Note that CNNs proposed in some literatures [2]-[4] are serial image processings with the single-layer CNN, and on the contrary, our
two-layer CNN is parallel image processings. Therefore,
compared to the single-layer CNNs, the two-layer CNNs
can more eﬃciently solve some problems in image processings, which can not be easily obtained by single-layer
CNNs.
III. TEMPLATES AND ALGORITHMS FOR
IMAGE PROCESSING
Image processing is one of the most important applications of the CNNs. In this section, we will give several
important and interested examples of the two-layer CNNs
which show eﬃciently for image processing compared tothe single-layer CNNs.
A. Linearly Non-Separable Problems
Now, let us consider a two-layer uncoupled CNN. For
simpliﬁcation, we omit the subscript (i,j), because eachcell in CNNs is coupled in the same way. Thus, the cell

0-7803-7278-6/02/$10.00 ©2002 IEEE

-1

Fig. 2. The trajectories of state variables for various g1 andg2 , where we set a1,00 = a2,00 = 2 and the stable andunstable equilibrium points are denoted by the solid dots
and circles, respectively.

equations are written as follows;
ẋ1
ẋ2

=
=
=
=


−x1 + a1,00 y1 + b1,00 u1 + c12,00 y2 + I1 
−x1 + a1,00 y1 + g1
(4)−x2 + a2,00 y2 + b2,00 u2 + c21,00 y1 + I2 
−x2 + a2,00 y2 + g2

For the self-feedback coeﬃcients a1,00 > 1 and a2,00 > 1,
the trajectory of the state variables without g1 and g2
behave as shown by the solid line in Fig.2. By shifting
g1 and g2 , we have the trajectries shown by the dottedlines in the ﬁgure. Observe that if we choose the initial conditions x1 (0) and x2 (0) larger than the unstableequilibrium points, the steady-state output will be given
by 1, and otherwise, the outputs given by -1. Thus, wehave the following output relations depending the initialconditions;
g1
y1 = sgn[x1 (0) + a1,00

−1 ]


b1,00 u1 +c12,00 y2 +I1]= sgn[x1 (0) +a1,00 −1
(5)
g2
=
sgn[x
(0)
+
]
y

2
2
a2,00 −1



b
u +c
y1 +I2= sgn[x2 (0) + 2,00 2a2,0021,00]−1
where b1,00 , b2,00 , c12,00 , c21,00 , I1 and I2 are the constantparameters. u1 and u2 are the inputs of cells, which areusually equal to 1 or −1 corresponding to black or white
pixel in the binary image, or are between 1 and −1 for thegrey scale image. Therefore, by suitably selecting theseparameters and initial state conditions, we can obtain
the expected outputs.
For example, consider the two-layer CNN to performa linearly non-separable task – logic XOR, which cannot
be directly solved by the single-layer CNNs. The logic
function can be written as
F = X1 ⊕ X2

(6)

(a)

(b)

(c)

(a)

(b)

(d)

(e)

(f)

(c)

(d)

Fig. 3. An example for solving a non-linearly separable problem. (a), (b) and (c) are the input, the initial state condition and the output of first layer, (d), (e) and (f) are
the input, the initial state condition and the output ofsecond layer, respectively.

Fig. 4. An example for dividing object into half-and-half. (a)the input of first layer, (b) the output of first layer, (c)the input of second layer, (d) the output of second layer.

can execute some complicated tasks.
The input of two pixels has four possible combinations.
The output will be black only if one of the two pixels isblack. The templates are:
A1 = A2 = 2, B1 = B2 = 1,
C21 = −2, C12 = 0, I1 = I2 = −1.

(7)

One of the two given images is set to the input u1 of ﬁrst
layer CNN and the initial state x2 (0) of the second layer.
The another image is set the input u2 of second layer andthe state x1 (0) of ﬁrst layer. The execution result canbe obtained from the output of second layer. They are
shown by Fig.3, where (a) and (d) are two input images of
the two layers, and (b) and (e) are the initial state. The
execution result is shown by (f), where the black pixels
are obtained only when one of the two inputs has a black
pixel, and white pixels are obtained for every other cases.
The output of ﬁrst layer shows the result obtained bylogic AND of the two images. These results are calculatedby the relations (5). From the above simulation result,
we have shown that the uncoupled two-layer CNNs arecapable to perform non-linearly separable tasks.
In fact, the above example is executed based on the twolayers of the CNN with cascade. Moreover, in this case,
we can see that not only the developed templates andstability conditions in single-layer CNNs are valid to the
two-layer CNNs, but also some compound tasks, which
consist of logic operations, edge detection, etc., can bedirectly solved in one step. When the two C templates
are not zero, the two-layer CNN constitutes a closedloop case, every layer output is fed into the other layer
input. It is because we use the mutual feed characteristics
between the two layers of the CNN, the two-layer CNNs

0-7803-7278-6/02/$10.00 ©2002 IEEE

B. Divide Object into Half-and-half
Now let us consider a problem to divide object into halfand-half or two parts with the same area. This task
can be executed by the two-layer CNN in one step. The
basic idea of the algorithm is that, the two layers of the
CNN are respectively and simultaneously used to peel
pixels from two sides of the object, when the processreaches the centerline, the two-layer CNN arrives at thesteady state, thus the object is divided into two parts
which are respectively output in the CNN two layers.
The templates are given as follows;
A1 =1 2 −1 , B1 = 0 1 1 , I1 = −2

A2 =−1 2 1 , B2 = 1 1 0 , I2 = −2

C12 = 1 0 −1 , C21 =−1 0 1 ;(8)

In this case, the given binary image is set to all the inputs
and initial states of the CNN as shown in Fig.4(a) and(c). Thus, we obtained the two diﬀerent parts with thesame area from the given object on the two layer outputsof the CNN, which are shown in Fig.4(b) and (d).
C. Center Point Detection
It is very important to detect the center point of a given
object. Unfortunately, the deﬁnition of the center pointis very ambiguous, because there are various objects suchas convex, concave, disk with embeded holes, etc. In
this paper, we deﬁne “the center point is located at the
halfway from the furthermost points of a given object”.
Therefore, Without the loss of generality, we can consider that the object image is a rectangle block such as

(a)

(e)

(b)

(f)

(c)

(g)

(d)

(h)

Fig. 5. An example for extracting the center line of an object.
States from (a) to (d) are the output of the first layer inthe time progress, and those from (e) to (h) are the output
of second layer. Final results of the two-layer CNN are
shown by (d) and (h).

Fig.5(a), because arbitrary shape objects can be changed
into rectangle blocks with the same length and width byusing the shadow template [5]. If a single-layer CNN isapplied to this problem, the algorithm is based on 8 steps
or 8 single-layer CNNs [5]. The eight single-layer CNNs
are circularly and continuously used to peel oﬀ pixels
from the object in eight directions until only the center
point of the object is remained. This is a very complicated process. In contrast, by using the two-layer CNNs,
this task can be solved in two steps, the ﬁrst step is todetect the centerline of the object, and the second stepis to detect the center point of the above centerline.
C.1 Detection of the Center Line of an Object
Now, consider a center line detection problem shownin Fig.5(a). The two layers of the CNN are respectively
used to continuously peel oﬀ the most left side and/or
right side pixels of the object at the same time, until its
centerline is left. For this purpose, we set the object image to the initial states of the both CNN layers as shown
Fig.5(a) and (e), and adopt the following templates:




0 0 0
0 0 0
B = 0,
A1= 0.5 2 −0.5  , C12= 0 2.8 −1  , 1
I1 = −2,
0 0 0
0 0 0




0 0 0
0 0 0
B = 0,
A2= −0.5 2 0.5 , C21= −1 2.8 0 , 2
(9)
I2 = −2,
0 0 0
0 0 0
At the beginning of this simulation, the ﬁrst layer andthe second layer of the CNN respectively perform apeeling-oﬀ the leftmost pixels and the rightmost pixelsof the object. Thus we get the middle results as shownin Fig.5(b) and (f), respectively. After that, the rightmost pixels of the ﬁrst layer and the leftmost pixels of thesecond layer are peeled oﬀ, then we get the next middle

0-7803-7278-6/02/$10.00 ©2002 IEEE

Fig. 6. The sum of state error via time for center line extraction

results as shown in Fig.5(c) and (g), respectively. As we
can see that Fig.5(c) and (g) are the same with Fig.5(a)and (e), except for the both end pixels which have been
peeled oﬀ. Thus, one cycle of the peeling-oﬀ is completed.
Like this cycle continues until the vertical centerline is remained. The transient has arrived at the steady state.
The two-layer stable outputs are shown in Fig.5(d) and(h), respectively. So far we obtain the center line of thethe object. In this simulation, the zero-ﬁxed boundary
condition is also adopted. This dynamic process can beobserved from the graph of the sum of state-errors varying with the integration time shown in Fig.6. The sumof state-errors is deﬁned as
SCN N (t) =

M 
N


((x1,ij (t + ∆t) − x1,ij (t))2i=1 j=1+(x2,ij (t + ∆t) − x2,ij (t))2 )(10)

The valleys show the states when the pixels from the twolayers are unbalanced states. After the object is completely peeled oﬀ and becomes a line, the sum of stateerrors decays to zero.
C.2 Detection of the Center PointTo detect the center point from the above center line,
we need to apply the same algorithm to the center linewith the transposed templates as follows;
0 0.5 00 0 0
B = 0,
A1= 0 2 0  , C12= 0 2.8 0  , 1
I1 = −2,0 −0.5 0
0 −1 0
0 −0.5 0
0 −1 0
B = 0,
A2= 0 2 0 , C21= 0 2.8 0 , 2
(11)
I2 = −2,
0 0.5 00 0 0The simulation results are shown in Fig.7. Thus, we canﬁnd the center point in two steps.
D. Skeletonization
Skeletonization is a very important task in image processing. The problem of skeletonization using CNN has

(a)

(b)

(c)

(a)

(b)

Fig. 8. An example for skeletonization. (a) is the initialstates, (b) is stable output of the second layer.

(d)

(e)

(f)

Fig. 7. An example for extracting the center point of line. (a),(b) and (c) are the initial state condition, the transient
result and the output of first layer. (d), (e) and (f) are
the initial state condition, the transient result and theoutput of the second layer, respectively.

already attacked in several literatures [2]-[4],etc. Similar
to center point dectection, all of the solutions are also
based on using complex multilayer, time variant templates or sometimes even nonuniform CNN architectures.
similarly, by using the two-layer CNN, the problem canbe eﬃciently solved. The algorithm is described in thefollowing.
The templates are




0 00
0.07 0.07 0.07
A1= 0 4 0 , C12= 0.07 −1.6 0.07 ,
0 00
0.07 0.07 0.07




0 0 0
−0.5 −0.5 −0.5
A2= 0 3.1 0 , C21= −0.5 4.2 −0.5 ,
0 0 0−0.5 −0.5 −0.5
B1 = B2 = 0, I1 = I2 = −1.9,(12)

For analyzing the convergence properties of dynamic
nonlinear systems, one of the most eﬀective techniques is
Lyapunov’s method, which has been successfully applied
to the stability analysis of single-layer CNN [1]. In thissection, we analyze simply the stability of the two-layer
CNN with the same techniques. We deﬁne a Lyapunov
function E(t) of a two-layer CNN by the scalar function,
which is similar to the one used by Hopﬁeld in [5] and canbe interpreted as the “generalized energy” for a two-layer
CNN.

E(t) = − 12 (i,j) (k,l) A1 (i, j; k, l)y1,ij y1,kl+ (i,j) (k,l) B1 (i, j; k, l)y1,ij u1,kl− 12 (i,j) (k,l) C1 (i, j; k, l)y1,ij y2,kl2+ 12 (i,j) y1,ij− (i,j) I1 y1,ij(13)− 12 (i,j) (k,l) A2 (i, j; k, l)y2,ij y2,kl+ (i,j) (k,l) B2 (i, j; k, l)y2,ij u2,kl− 12 (i,j) (k,l) C2 (i, j; k, l)y2,ij y1,kl2+ 12 (i,j) y2,ij− (i,j) I2 y1,ij
Therefore, we can show that the function E(t) isbounded as follows;

Where
Emax

In this simulation, the binary image shown in Fig.8(a)are set to each layer initial states of the CNN. The inputs
are arbitrary and the boundary condition is zero-ﬁxed.
Thus, we obtained the result from the second layer stable
output as shown in Fig.8(b). We have shown the powerfulness of the two-layer CNN for skeletonization, eventhough there are a few inperfections yet in the second
output.
IV. Stability of Two-Layer CNNs
Since the CNNs are usually required to be stable forthe applications, the studies on complete stability havebeen vigorously discussed and many criteria have beenobtained. In this section, we will discuss the convergenceproperty, and its related problems for the two-layer CNNs
with symmetric and/or special templates.

0-7803-7278-6/02/$10.00 ©2002 IEEE=

Maxt |E(t)| ≤ Emax

(14)



+ 21 (i,j) (k,l) |A1 (i, j; k, l)|+ (i,j) (k,l) |B1 (i, j; k, l)|+ 21 (i,j) (k,l) |C1 (i, j; k, l)|+ 21 (i,j) (k,l) |A2 (i, j; k, l)|+ (i,j) (k,l) |B2 (i, j; k, l)|+ 21 (i,j) (k,l) |C2 (i, j; k, l)|+M N (1 + |I1 | + |I2 |)(15)

Now, we consider two cases having the following cloning
templates;•

Case 1A1 (i, j; k, l) = A1 (k, l; i, j)
A2 (i, j; k, l) = A2 (k, l; i, j)
C1 (i, j; k, l) = C1 (k, l; i, j) = C(i, j; k, l)C2 (i, j; k, l) = C2 (k, l; i, j) = C(i, j; k, l)

(16)

•

Case 2A1 (i, j; k, l) = A1 (k, l; i, j)
A2 (i, j; k, l) = A2 (k, l; i, j)
C1 (i, j; k, l) = C2 (k, l; i, j)
C2 (i, j; k, l) = C1 (k, l; i, j)(17)

From Eq.(13), we have the following relation for thecases of 1 and 2:
dy1,ij 2
dy2,ij 2dE(t)(18)(i,j) [( dt ) + ( dt ) ] ≤ 0
dt = −
Thus, we found that the energy function is monotone
decreasing, where we used the following constraint conditions;dy1,ij1 |x1,ij | < 1=(19)
0 |x1,ij | ≥ 1
dx1,ijdy2,ij=dx2,ij

1
0|x2,ij | < 1|x2,ij | ≥ 1(20)

We ﬁnd that from Eq.(14) and (18), for any given inputs
u1 , u2 and the initial states x1 , x2 , we obtain thatlim E(t) = const.

t→∞

(21)

and

dE(t)
=0
(22)
dt
Thus, in the cases of 1 and 2, we always have the stablesteady state outputs in both two layers after the transient. Now, we show that both the states x1 and x2
approaches to the equibrium points. Let us consider thesystem equations Eq.(1)-(3) again, and rewrite them inthe following form;
lim

t→∞

dx1,ij (t)= −f1 (x1,ij (t)) + g1 (t)dtdx2,ij (t)= −f2 (x2,ij (t)) + g2 (t)
dt

(23)
(24)

where
1
f1(x1,ij ) = x1,ij − A1 (i, j; i, j)(|x1,ij +1|−|x1,ij −1|) (25)21
f2(x2,ij ) = x2,ij − A2 (i, j; i, j)(|x2,ij +1|−|x2,ij −1|) (26)2A1 (i, j; k, l)y1,kl + C21 (i, j; k, l)y2,kl
g1 (t) =(k,l)=(i,j)+
g2 (t) =





(k,l)

B1 (i, j; k, l)u1,kl + I1 (27)

(k,l)


A2 (i, j; k, l)y2,kl +C12 (i, j; k, l)y1,kl(k,l)=(i,j)

+



(k,l)

B2 (i, j; k, l)u2,kl + I2 (28)(k,l)

0-7803-7278-6/02/$10.00 ©2002 IEEE

Observe that g1 (t) and g2 (t) are not related to the states
x1,ij and x2,ij , and are only the functions of the outputs(y1 , y2 ), inputs (u1 , u2 ) and biases (I1 , I2 ). Therefore,
for A1 (i, j; i, j) > 1 and A1 (i, j; i, j) > 1 the trajectries
have the same structures as shown in Fig.2, and havethe stable and unstable equibrium points. Thus, we canconclude for the cases of 1 and 2 that the steady state of
two-layer CNN is completely stable, in the meaning thatboth states and outputs are in the steady state. The
validity of the above analysis are comﬁrmed from theabove simulation examples.
V. Conclusions
In this paper, we preliminarily investigated the image processing applications and stability of the two-layer
CNNs. Although the examples proposed in this paper
can be solved by single-layer CNNs, the applications of
two-layer CNNs to these problems are very eﬃcient compared to the single-layers. Especially, the center point
detection and skeletonzation problems with the two-layer
CNN can be found the solution in simple methods, however, if we use the single-layer CNN, it will be found many
steps depending on the size. We can prove the stability
for the special cases of two-layer CNNs, where its necessary conditions are proved by the Lyapunov function
and behaviors of the trajectories on the phase plane.
References[1] L.O. Chua and L. Yang, “Cellular neural networks: theory andapplications,” IEEE Trans. Circuits & Syst., vol.35 pp.1257–
1290, Oct. 1988.
[2] P.L. Ventianer, F. werblin, T. Roska and L.O. Chua, “Analogic
CNN algorithms for some image compression and restoratintasks,” IEEE Trans. Circuits & Syst., vol.42, pp.278–284, Mar.
1995.
[3] H. Harrer and J. A. Nossek, “Skeletonization: A new application for discrete-time cellular neural networks using timevariant templates,” in proc. IEEE Int. Symp. Circuits & Syst.,
1992, pp.2897-2900.
[4] T. Roska, L. Kek, L. Nemes, A. Zarandy, and P. Szolgay, “CNN
Software Libarary (Templates and Algorithms), Version 7.3,”
Analogical and neural computing laboratory, Computer andAutomation Institute of the Hungarian Academy of Sciences,
Budapest, Hungary, 1999.
[5] J. J. Hopfield, “Neurons with graded response have collective
computational properties like those of two-state neurons,” inProc. Natl. acad. sci USA, vol.81, pp.3088-3092,1984.