An Improvement of the Design Method of Cellular
Neural Networks Based on Generalized Eigenvalue
Minimization
Author(s): Ryoma Bise and Norikazu Takahashi
Journal: IEEE Transactions on Circuits and Systems I
Volume: 50
Number: 12
Pages: 1569–1574
Month: December
Year: 2003
Published Version: http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=1257462

c⃝2003
IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing thismaterial for advertising or promotional purposes, creating new collective works, for resale orredistribution to servers or lists, or reuse of any copyrighted component of this work in otherworks.

1

An Improvement of the Design Method of
Cellular Neural Networks based on Generalized
Eigenvalue Minimization
Ryoma Bise , Norikazu Takahashi, and Tetsuo Nishi
Abstract— Realization of associative memories by cellular neural networks (CNNs) with binary output is studied. Concerning
this problem, a CNN design method based upon generalized eigenvalue minimization (GEVM) has recently been proposed. In thispaper, a new CNN design method which is based on the GEVMbased method will be presented. We first give some analytical results related to the basin of attraction of a memory vector. We thenderive the design method by combining those analytical results
and the GEVM-based method. We finally show through computer
simulations that the proposed method can achieve higher recall
probability than the original GEVM-based method.
Index Terms— Cellular neural networks, associative memory,basin of attraction, generalized eigenvalue minimization.

I. I NTRODUCTION
A cellular neural network (CNN) is a nonlinear analog circuit
consisting of a number of locally coupled signal processing elements called cells. Since the first paper of Chua and Yang [1]was published in 1988, CNNs have found many applications
mainly in the field of image processing. In each application of
CNNs, it is very important to find optimum values of the network parameters so that a CNN performs a desired task. So far,there have been many attempts to construct systematic ways ofdesigning CNNs with space-invariant couplings for image processing tasks [2], [3], [4], [5].
In this paper, as a fundamental design problem for CNNs,
we consider the realization of associative memories by meansof CNNs with space-varying couplings. As is well known, thisproblem has been vigorously studied for fully coupled neuralnetworks such as the Hopfield model from the early ninetyeighties, and various design methods have been proposed so
far [6], [7], [8], [9], [10], [11]. However, since each cell in a
CNN is connected only with its neighboring cells, these methods cannot be applied directly to CNNs. In order to makeuse of CNNs for associative memories, it is thus required todevelop design methods suitable for their structural characteristics. Liu and Michel [12] have proposed a design method
for sparsely interconnected neural networks and applied it to
CNNs. Their method is a modification of the eigenstructure
method [10] which is well known as an effective design technique for fully connected neural networks. Seiler et al. [13] developed an optimization-based method for realizing prescribedstable output patterns and unstable output patterns of a CNN.
The method proposed by Grassi [14], [15] is distinct from others because it makes use of CNNs possessing a unique equilibrium point which is globally asymptotically stable and input
patterns are fed into such CNNs as bias vectors.
Park et al. [16] have recently proposed a method to design CNNs based on the generalized eigenvalue minimization(GEVM) [17], [18], [19]. Throughout this paper, this method
will be referred to as Park’s method for simplicity. In the synthesis procedure of Park’s method, for each step the feasibilityof a set of linear inequalities is first checked. If it is feasible, a
GEVM problem is solved to find the network parameters whichguarantee both that the prototype vectors are stored as memory
vectors and that the basin of attraction of each prototype vector is maximized in a certain sense. Otherwise, a linear matrix
inequality (LMI) problem is solved to find the network parameters which guarantee just that the prototype vectors are stored as
memory vectors. Since both GEVM problems and LMI problems are solved efficiently by using computer software such as
MATLAB [20], the network parameters can be easily obtained.
The results of computer simulations carried out by Park et al.
[16] have shown that Park’s method is superior in the average
recall probability to the modified eigenstructure method proposed by Liu and Michel. Therefore, Park’s method can be
regarded as one of the most effective CNN design techniques
for associative memories. We note here that LMIs also play
important roles in the stability analysis of recurrent neural networks; for example, LMI conditions are given by Suykens et al.
[21], [22] in relation to the global asymptotic stability of multilayer recurrent neural networks and the basins of attraction ofequilibrium points.
In this paper, we propose a new CNN design procedure basedon Park’s method. We first give some analytical results related
to the basin of attraction of a memory vector. One of them isa generalization of a theorem given by Park et al. [16], andthe others are firstly obtained in the present paper. We thendevelop the CNN design procedure by modifying a certain partof Park’s method by using those analytical results. We finallyshow through computer simulations that the proposed method
can achieve higher recall probability than the original GEVMbased method.
II. P ROBLEM F ORMULATIONLet us consider CNNs described by the following differential
equations:∑dxi= −xi +Aij yj + Ii ,dt

i = 1, 2, . . . , n

(1)

j∈N̄i

A previous version of this paper appeared in the Proceedings of the 7th IEEE
International Workshop on Cellular Neural Networks and their Applications,pp.515–522, 2002. This research was partially supported by Japan Society for
the Promotion of Science, Grant-in-Aid for Encouragement of Young Scientists, 15760268, 2003 and the 21st Century COE Program ‘Reconstruction ofSocial Infrastructure Related to Information Science and Electrical Engineering’.
R. Bise is with the Dai Nippon Printing Co. Ltd., Tokyo, Japan (email:Bise-R@mail.dnp.co.jp)
N. Takahashi and T. Nishi are with the Department of Computer Science
and Communication Engineering, Kyushu University, Fukuoka, Japan (email:norikazu@csce.kyushu-u.ac.jp; nishi@csce.kyushu-u.ac.jp)

where xi is the state of the i-th cell, yi the output of the i-th cell
determined by xi throughyi = f (xi ) ,1(|xi + 1| − |xi − 1|),
2

(2)

Aij the coupling coefficient from the j-th cell to the i-th cell,Ii the bias of the i-th cell, and N̄i the set of indices of the cells
belonging to the neighborhood of the i-th cell. Although it is2

often assumed in CNN literature that coupling coefficients between cells are space-invariant [1], we will not make this assumption in this paper. Note that the sets N̄i , i = 1, 2, . . . , n
are uniquely determined by the method of numbering cells andthe radius of neighborhood denoted by r. In the following, theneighborhood of the i-th cell excluding itself is represented by
Ni , that is, Ni , N̄i \ {i} for convenience. Also, the set of ndimensional real vectors, the set of n × n real matrices, and theset of n dimensional binary vectors, i.e. the vectors whose elements are +1 or −1 are denoted by Rn , Rn×n and Bn , respectively. By introducing the state vector x = [x1 , x2 , . . . , xn ]T ,the output vector y = [y1 , y2 , . . . , yn ]T , the connection matrix
A = [Aij ] ∈ Rn×n , the bias vector I = [I1 , I2 , . . . , In ]T , thepiecewise linear mapping f (x) , [f (x1 ), f (x2 ), . . . , f (xn )]T ,and the set of matrices M (N̄1 , N̄2 , . . . , N̄n ) defined by
M (N̄1 , N̄2 , . . . , N̄n ){}, S = [Sij ] ∈ Rn×n | Sij = 0 if j ̸∈ N̄i ,we can rewrite (1) and (2) in vector form as follows:dx= −x + Ay + I
dt(3)

y = f (x)

(4)

where
the
connection
matrixA
belongstoM (N̄1 , N̄2 , . . . , N̄n ).
A vector y e is referred to as amemory vector of the CNN described by (3) and (4) if the
CNN has an asymptotically stable equilibrium point xe suchthat y e = f (xe ). The set of initial states x(0) such that
limt→∞ x(t) = xe is called the basin of attraction of the memory vector y e . As is well known, if every diagonal element of
A is greater than or equal to unity, then asymptotically stableequilibrium points can exist only in the total saturation region
which is defined by {x ∈ Rn | |xi | ≥ 1, ∀i} [1], [23]. Thus, inthis case, all memory vectors are binary.
Based on the design problem given by Michel and Liu [11]for fully coupled neural networks, we formulate the CNN design problem for associative memories as follows:
CNN Design Problem:
For given prototype vectors
α1 , α2 , . . . , αm ∈ {1, −1}n and the sets N̄1 , N̄2 , . . . , N̄n , findthe connection matrix A ∈ M (N̄1 , N̄2 , . . . , N̄n ) and the bias
vector I such that the synthesized CNN has the following properties.
1) All prototype vectors α1 , α2 , . . . , αm are memory vectors.
2) The total number of spurious memory vectors, thatis, the memory vectors of the CNN not contained in{α1 , α2 , . . . , αm }, is as small as possible.
3) The basin of attraction of each prototype vector is as large
as possible.
4) The CNN has no oscillatory solution.
Since we will not assume that the connection matrix A issymmetric or satisfies other stability conditions obtained so far[24], [25], [26], the fourth property is not satisfied in general.
We will thus focus our attention only on the first three properties. Note here that the second and third properties are closely
related because the basins of attraction of the prototype vectors

become larger, as the total number of spurious memory vectors
becomes smaller, and vice versa. Note also that there will bevarious strategies for designing CNNs depending on what kindof qualitative criterion for the sizes of the basins of attraction
is considered. For example, Park’s method designs a CNN sothat the smallest basin of attraction is maximized in a certain
sense. In this case, the design problem is formulated as a kindof minmax problem.
III. A NALYSIS
In this section, we will present some theorems concerning
the basin of attraction of a memory vector of a CNN whichplay important roles in the CNN design method given in thenext section.
Theorem 1: Suppose n sets N̄1 , N̄2 , . . . , N̄n and a binary
vector α∗ = [α1∗ , α2∗ , . . . , αn∗ ]T ∈ Bn are given. If the connection matrix A = [Aij ] ∈ M (N̄1 , N̄2 , . . . , N̄n ) and the bias
vector I satisfy


∑
αi∗ 
Aij αj∗ + Ii  > κi max |Aij | + (Aii − 1) (5)
j∈Ni

j∈Ni

with Aii ≥ 1 and κi ≥ 0, then any vector β =Tn∗[β∑1 , β2 , . . . , βn ] ∗ ∈ R such that f (βi ) ̸= αi andj∈Ni |f (βj ) − αj | ≤ κi has the following properties.
1) The vector α = f (β) is not a memory vector.
2) If x(0) = β then xi (t) moves toward αi∗ at t = 0.
Proof: It follows from the state equation of a CNN that


∑
dx
i
= αi∗ −βi +Aij f (βj ) + Ii αi∗ ·dt x=βj∈N̄i∑Aij f (βj ) + Ii  − αi∗ βi . (6)= αi∗ j∈N̄i

By applying the conditions to the first term on the right-handside, we have


∑
αi∗ 
Aij f (βj ) + Ii 
j∈N̄i



= Aii αi∗ f (βi ) + αi∗ ∑Aij f (βj ) + Ii j∈Ni

= Aii αi∗ f (βi ) + αi∗
+

∑

(∑

Aij αj∗ + Ii

j∈Ni

Aij (f (βj ) − αj∗ )

j∈Ni



≥ Aii αi∗ f (βi ) + αi∗ )∑Aij αj∗ + Ii 

j∈Ni

−

∑
j∈Ni

Aij (f (βj ) − αj∗ )3≥ Aii αi∗ f (βi ) + αi∗ 

∑





Aij αj∗ + Ii  − κi max |Aij |
j∈Ni

j∈Ni



∑

= (1 + αi∗ f (βi )) Aii − 1 + αi∗ ≥ αi∗ Aij αj∗ + Ii = αi∗ j∈Ni

(7)

From (6) and (7) the following inequality is obtained.
dxi> αi∗ f (βi ) − αi∗ βidt x=β(8)Since it is assumed that f (βi ) ̸= αi∗ , there are two possible
cases where |f (βi )| < 1 and f (βi ) = −αi∗ . In the former case,the right-hand side of (8) vanishes because f (βi ) = βi holds.
In the latter case, the right-hand side of (8) takes a nonnegative value because αi∗ f (βi ) = −1 ≥ αi∗ βi holds. Therefore,(6) always takes a positive value. This means that β is not an
equilibrium point and that if x(0) = β then xi (t) tends to αi∗
at t = 0.
Although Theorem 1 is very similar to a theorem given byPark et al. [16], these two results are different at the following two points. One is that Theorem 1 contains the condition
Aii ≥ 1 while in the theorem of Park et al. no condition isassumed on the value of Aii . Since, as far as only binary output is concerned, it is common in designing CNNs to assume
Aii ≥ 1, adding this condition will not lose generality. In fact,
Aii ≥ 1 is assumed even in Park’s method. The other is that β
is assumed to be a real vector in Theorem 1 while β is restricted
to a binary vector in the theorem of Park et al.. Because of thisdifference, Theorem 1 can be considered as an extension of thetheorem of Park et al..
Theorem 2: Suppose n sets N̄1 , N̄2 , . . . , N̄n and a binary
vector α∗ = [α1∗ , α2∗ , . . . , αn∗ ]T ∈ Bn are given. If the connection matrix A = [Aij ] ∈ M (N̄1 , N̄2 , . . . , N̄n ) and the bias
vector I satisfy


∑
αi∗ 
Aij αj∗ + Ii  > κi max |Aij | − (Aii − 1) (9)withvector β ∈ Rn such that f (βi ) ̸= αi∗ and∑ κi ≥ 0, then any
∗
|f
(β
)
−
α
|
≤
κi has the followings properties.
j
j
j∈N̄i
1) The vector α = f (β) is not a memory vector.
2) If x(0) = β then xi (t) moves toward αi∗ at t = 0.
Proof: By applying the condition to the first term on theright-hand side of (6), we have


∑
αi∗ 
Aij f (βj ) + Ii = αi∗ 

j∈N̄i

j∈N̄i

∑


Aij αj∗ + Ii  − κi max |Aij | + (Aii − 1) + 1

∑
j∈N̄i

Aij αj∗ + Ii +j∈N̄i

The rest of the proof is same as Theorem 1.
The following theorem follows from Theorems 1 and 2.
Theorem 3: Let I1 and I2 be two sets such that I1 ∪ I2 ={1, 2, . . . , n} and I1 ∩ I2 = ϕ. Suppose n sets N̄1 , N̄2 , . . . , N̄nand a binary vector α∗ = [α1∗ , α2∗ , . . . , αn∗ ]T ∈ Bn are given. If
the connection matrix A ∈ M (N̄1 , N̄2 , . . . , N̄n ) and the bias
vector I of a CNN satisfy (5) with Aii ≥ 1 and κi ≥ 0 for alli ∈ I1 , and satisfy (9) with κi ≥ 0 for all i ∈ I2 , then α∗ is amemory vector of the CNN and any vector β ∈ Rn satisfying∑|f (βj ) − αj∗ | ≤ κi , ∀i ∈ I1 


j∈N
∑i
(10)
|f (βj ) − αj∗ | ≤ κi , ∀i ∈ I2 


j∈N̄i

belongs to the basin of attraction of α∗ .
Proof: We first prove that α∗ is a memory vector. Since(5) is satisfied with Aii ≥ 1 and κi ≥ 0 for any i ∈ I1 , we have




∑
∑
Aij αj∗ + Ii 
Aij αj∗ + Ii  = Aii + αi∗ 
αi∗ 
j∈Ni

j∈N̄i

> Aii + κi max |Aij | + (Aii − 1)j∈Ni

≥ 2Aii − 1≥1
for all i ∈ I1 . Since (9) is satisfied with κi ≥ 0 for any i ∈ I2 ,we also have




∑
∑
αi∗ 
Aij αj∗ + Ii  = Aii + αi∗ 
Aij αj∗ + Ii 
j∈Ni

j∈N̄i

> Aii + κi max |Aij | − (Aii − 1)
j∈N̄i

≥1

j∈N̄i

j∈Ni

j∈N̄i

Aij αj∗ + Ii  − κi max |Aij |> 1≥ αi∗ f (βi ).

j∈Ni(1 + αi∗ f (βi ))Aii − 1
αi∗ f (βi ).

αi∗ ·



j∈Ni

−κi max |Aij | − (Aii − 1)
>
≥

∑

∑
j∈N̄i


Aij (f (βj ) − αj∗ )

for all i ∈ I2 . It follows from the above two inequalities that


∑
αi∗ 
Aij αj∗ + Ii  > 1, i = 1, 2, . . . , nj∈N̄iwhich means that there exists an equilibrium point x∗ such thatf (x∗ ) = α∗ . Since this equilibrium point lies inside the total
saturation region, it is asymptotically stable. Therefore, α∗ isa memory vector of the CNN. Next, we will prove the second
statement of the theorem. Let R be the set of vectors β ∈Rn satisfying (10). It follows from Theorems 1 and 2 that ifx(t0 ) ∈ R thend|f (xi (t)) − αi∗ | ≤ 0,
dt

i = 1, 2, . . . , n

(11)

4

holds at t = t0 . The condition (11) holds with an equal sign if
and only if f (xi (t0 )) = αi∗ . As a consequence, if x(0) ∈ R
then x(t) ∈ R for all t ≥ 0 and f (x(t)) converges to α∗
monotonically.
IV. D ESIGNIn the CNN design problem given in Section II, the most important thing is to store m prototype vectors α1 , α2 , . . . , αm as
memory vectors. This is achieved by choosing the connection
matrix A ∈ M (N̄1 , N̄2 , . . . , N̄n ) and the bias vector I suchthat the set of inequalities


∑
αik 
Aij αjk + Ii  > 1, k = 1, 2, . . . , m
(12)
j∈N̄i

holds for i = 1, 2, . . . , n. Note that (12) is feasible with
Aii = 1 + ε (ε > 0) for any set of the prototype vectors
α1 , α2 , . . . , αm because the left-hand side of (12) becomes
1 + ε if we put Aij = 0, ∀j ∈ Ni and Ii = 0. It is also
important in the CNN design problem to guarantee that memory vectors are restricted to be binary. As we have mentioned
before, this is achieved by choosing the values of the diagonal
elements of A such thatAii ≥ 1,i = 1, 2, . . . , n.

(13)

The main idea of Park’s method is to try to make the basinsof attraction of the prototype vectors as large as possible bymaking use of Theorem 1 while guaranteeing the above two requirements, i.e., (12) and (13). The total number of spurious
memory vectors is expected to be reduced if the basins of attraction become large. Let us consider the set of inequalities


∑
k
k
αi
Aij αj + Ii  > κi max |Aij | + (Aii − 1),
j∈Ni

j∈Ni

k = 1, 2, . . . , m(14)which corresponds to (5) in Theorem 1. If κi can be maximized under the constraints (14), κi ≥ 0 and Aii ≥ 1 for
i = 1, 2, . . . , n, then the basins of attraction of the prototype
vectors are maximized in some sense. Note here that κi is maximized when Aii = 1. Note also that (14) is feasible with κi ≥ 0
and Aii = 1 if and only if


∑
αik 
Aij αjk + Ii  > 0, k = 1, 2, . . . , m
(15)
j∈Ni

is feasible. Furthermore it is apparent that (12) is feasible with
Aii = 1 if and only if (15) is feasible. Park’s method thus firstchecks the feasibility of (15). If (15) is feasible, Aii is set to 1
and the optimization problem:
Minimize−κi

Subject to −κi qi + αik ∑Aij αjk + Ii  > 0,j∈Ni

k = 1, 2, . . . , m|Aij | < qi , ∀j ∈ Ni
|Ii | < U
L < qi < U.

(16)

is solved, where L and U (L < U ) are positive constants specified by users. The optimization problem (16) is in the form of
the GEVM problem. If, on the other hand, (15) is not feasible,
since (14) is not feasible with κi ≥ 0 and Aii ≥ 1 in this case,Aii is set to 1 + ε (ε > 0) and the values of Aij , j ∈ Ni and Ii
satisfying (12) with Aii = 1 + ε are found. One can easily seethat this is an LMI problem.
Although computer simulation results [16] show that Park’s
method can achieve much higher average recall probability thanthe modified eigenstructure method [12], it is still insufficientbecause the basins of attraction of prototype vectors are nottaken into account at all in the case where (15) is not feasible. In order to solve this problem, we make use of Theorem 2
in addition to Theorem 1.
Let us consider the set of inequalities

αik 

∑


Aij αjk + Ii  > κi max |Aij | − (Aii − 1),
j∈N̄i

j∈Ni

k = 1, 2, . . . , m(17)which corresponds to (9) in Theorem 2. If we set Aii = 1 +ε (ε > 0), Aij = 0, ∀j ∈ Ni , Ii = 0 and κi = 0 thenthe left-hand side of (17) becomes 0, while the right-hand side
becomes −ε. This means that the set of inequalities (17) is
feasible for any set of prototype vectors α1 , α2 , . . . , αm under
the conditions that Aii ≥ 1 and κi ≥ 0. Therefore, even if(15) is not feasible, we can make the basins of attraction of the
prototype vectors as large as possible by setting Aii = 1 + ε
and solving the optimization problem
Minimize
Subject to−κi−κi qi + αik ∑Aij αjk + Ii  > −ε,j∈Nik = 1, 2, . . . , m(18)|Aij | < qi , ∀j ∈ Ni
|Ii | < U
L < qi < U
where L and U (L < U ) are positive constants specified byusers. The optimization problem (18) is in the form of the
GEVM problem.
From the above considerations, we derive the following CNN
design method.
CNN Design Method: Given n sets N̄1 , N̄2 , . . . , N̄n ⊆{1, 2, . . . , n} and m prototype vectors α1 , α2 , . . . , αm ∈ Bn ,execute the following procedure for i = 1, 2, . . . , n.
1) Check whether the set of inequalities (15) is feasible. If
it is feasible go to Step 2), otherwise go to Step 3).
2) Set Aii = 1, and find Aij , j ∈ Ni and Ii by solving the
optimization problem (16).
3) Set Aii = 1 + ε (ε > 0), and find Aij , j ∈ Ni and Ii bysolving the optimization problem (18).
The proposed method tries to maximize κi in Theorem 2even in the case where the set of inequalities (15) is not feasible, while Park’s method just tries to satisfy the condition (12).
It is thus expected from Theorem 3 that the proposed method

5Fig. 1. Prototype vectors

TABLE IC OMPARISON OF THE AVERAGE RECALL PROBABILITY PAV (d) BETWEEN PARK ’ S METHOD AND THE PROPOSED METHOD

radius of neighborhood
r=1
r=2
r=3

design method
Park’s method
Proposed method
Park’s method
Proposed method
Park’s method
Proposed method

can realize larger basins of attraction of the prototype vectors
than Park’s method. Moreover, since the optimization problems(16) and (18) are in the form of the GEVM problem, they canbe efficiently solved numerically, as in Park’s method, by using
interior-point algorithms.
Let us now consider the computational complexity of the proposed method. First, the GEVM problems (16) and (18) can besolved in polynomial time in the size of the problem, which isdetermined by the number of prototype vectors and the radius ofneighborhood, by using some efficient optimization techniques[18]. Second, the computation time of the proposed method is
proportional to the number of cells as far as both the numberof prototype vectors and the radius of neighborhood are fixed.
From these facts, we can conclude that the proposed method
can be solved in linear time in the number of cells, and in polynomial time in the number of prototype vectors and the radiusof neighborhood. It is thus expected that the proposed method
is applicable for CNNs with a large number of cells.
V. C OMPUTER S IMULATIONSIn order to verify efficiency of the proposed method, we apply Park’s method and the proposed method to the same set of
prototype vectors, and compare the performance of synthesized
CNNs in terms of the average recall probability [16] which isdefined as follows: Let αk be any prototype vector of a CNN.
The probability that the CNN converges to αk when the initialstate x(0) is randomly chosen from the binary vectors such thatthe Hamming distance between αk and them is d is called therecall probability of αk from the Hamming distance d, and isdenoted by P (αk , d). Moreover, the average of P (αk , d) overAverage recall probability Pav (d)
d=1
d=2
d=3
d=4
d=5
0.4623 0.2239 0.1109 0.0566 0.0288
0.5479 0.2969 0.1623 0.0879 0.0455
0.9082 0.8186 0.7257 0.6319 0.5369
0.9584 0.8898 0.8046 0.7080 0.6026
0.9812 0.9576 0.9347 0.9058 0.8651
0.9937 0.9766 0.9585 0.9298 0.8891

all the prototype vectors α1 , α2 , . . . , αm is referred to as theaverage recall probability from Hamming distance d, and is denoted by Pav (d).
The prototype vectors used in this experiment are 26 English
capital letters shown in Fig. 1. Each pattern has 49 (= 7 × 7)
pixels where black and white pixels represent +1 and −1, respectively. For each value of the radius of neighborhood (r = 1,
2 and 3), we have determined the network parameters by applying Park’s method and the proposed method to the prototype
vectors. In both design methods, GEVM problems were solved
by using the function “gevp” in MATLAB LMI Control Toolbox [20] with the same values of constants L = 1, U = 10 and
ε = 0.1.
The simulation results are shown in Table 1. Here we haveestimated the average recall probability Pav (d), d = 1, 2, . . . , 5
by investigating the final output for all possible initial states in
the case where d ≤ 2 and by investigating the final output for
randomly chosen 3,000 vectors from the possible initial states
for each prototype vector in the case where d ≥ 3. As shown in
Table 1, the average recall probability of the proposed method
is higher than that of Park’s method in all cases. From theseresults, we can conclude that the proposed method is in fact
superior to Park’s method and therefore regarded as an efficient
CNN design method for associative memories. Also, as shownin Table 1, the smaller the radius of neighborhood, the bigger
the difference between the proposed method and Park’s method.
This is because the smaller the value of r, the more frequently
the case where (15) is not feasible happens.

6

VI. C ONCLUDING R EMARKS
We have developed a CNN design method for associative
memories by modifying the optimization-based method recently proposed by Park et al. with some analytical results presented in this paper. The proposed method tries to maximize thebasins of attraction of the prototype vectors in any case, while
the method proposed by Park et al. tries just to store the prototype vectors as memory vectors if a certain condition is not satisfied. As the computer simulation results show, the proposed
method can achieve higher recall probability in all cases, and iseffective in particular for CNNs with a small neighborhood.
We finally give some comments on the circuit implementation and the robustness of the proposed method. In the implementation of a CNN on an analog chip, the values of the parameters obtained numerically by the proposed method are notrealized exactly due to inherent inaccuracy of analog devices. It
is thus expected that the average recall probability achieved by
a CNN chip is lower than that derived by computer simulations.
However, since the proposed method chooses the values of theparameters so that κi in (14) or (17) is maximized, the small
deviation of the parameter values will not affect those inequalities, that is, there will still exist a positive κi satisfying (14) or(17). In this sense, the proposed method is robust for parameter
deviations. Moreover, it is obvious that the proposed method
has a higher degree of robustness than Park’s method. Quantitative investigation of the effect of the parameter deviation on
the average recall probability is a future problem.
R EFERENCES[1] L.O. Chua and L. Yang, “Cellular neural networks: theory,” IEEE Trans.
Circuits Syst. I, vol.35, pp.1257–1272, Oct. 1988.
[2] J.A. Nossek, “Design and learning with cellular neural networks,” Int. J.
Circuit Theory Appl., vol.24, pp.15–24, Jan.–Feb. 1996.
[3] Á. Zarándy, “The art of CNN template design,” Int. J. Circuit Theory Appl.,
vol.27, pp.5–23, Jan.–Feb. 1999.
[4] M. Hänggi and G.S. Moschytz, “An exact and direct analytical method forthe design of optimally robust CNN templates,” IEEE Trans. Circuits Syst.
I, vol.46, pp.304–311, Feb. 1999.
[5] M. Gilli and P.P. Civalleri, “Template design methods for binary stable
cellular neural networks,” Int. J. Circuit Theory Appl., vol.30, pp.211–230,
Mar.–Jun. 2002.
[6] J.J. Hopfield, “Neural networks and physical systems with emergent collective computational abilities,” Proc. Natl. Acad. Sci., U.S.A., vol.79,
pp.2554–2558, Apr. 1982.
[7] L. Personnaz, I. Guyon, and G.Dreyfus, “Information storage and retrievalin spin-glass like neural networks,” J. Phys. Lett., vol.46, pp.L359–L365,
Apr. 1985.
[8] L. Personnaz, I. Guyon, and G.Dreyfus, “Collective computational properties of neural networks: New learning mechanisms,” Phys. Lev. A, vol.34,
pp.4217–4228, Nov. 1986.
[9] Y. Kamp and M. Hasler, Recursive Neural Networks for Associative Memory, Chichester: John Wiley & Sons, 1990.
[10] A.N. Michel and J.A. Farrell, “Associative Memories via Artificial Neural
Networks,” IEEE Control Syst. Mag., vol.10, pp.6–17, Apr. 1990.
[11] A.N. Michel and D. Liu, Qualitative Analysis and Synthesis of Recurrent
Neural Networks, New York: Marcel Dekker, 2002.
[12] D. Liu and A.N. Michel, “Sparsely interconnected neural networks for
associative memories with applications to cellular neural networks,” IEEE
Trans. Circuits Syst. II, vol.41, pp.295–307, Apr. 1994.
[13] G. Seiler, A.J. Schuler, and J.A. Nossek, “Design of robust cellular neuralnetworks,” IEEE Trans. Circuits Syst. I, vol.40, pp.358–364, May 1993.
[14] G. Grassi, “A new approach to design cellular neural networks for associative memories,” IEEE Trans. Circuits Syst. I, vol.44, pp.835–838, Sep.
1997.
[15] G. Grassi, “On discrete-time cellular neural networks for associative
memories,” IEEE Trans. Circuits Syst. I, vol.48, pp.107–111, Jan. 2001.

[16] J. Park, H.-Y. Kim, Y. Park, and S.-W. Lee, “A synthesis procedure for
associative memories based on space-varying cellular neural networks,”
Neural Networks, vol.14, no.1, pp.107–113, Jan. 2001.
[17] S. Boyd and L. El Ghaoui, “Method of centers for minimizing generalized
eigenvalues,” Linear Algebra and its Applications, vol.188, pp.63–111, Jul.
1993.
[18] Yu.E. Nesterov and A.S. Nemirovskii, “An interior-point method forgeneralized linear-fractional programming,” Mathematical Programming,
vol.69, pp.177–204, Jul. 1995.
[19] L. Vandenberghe and V. Balakrishnan, “Algorithms and software for LMI
problems in control,” IEEE Control Syst. Mag., vol.17, pp.89–95, Oct.
1997.
[20] P. Gahinet, A. Nemirovski, A.J. Laub, and M. Chilali, LMI control toolbox, Natick: The MathWorks Inc., 1995.
[21] J.A.K. Suykens, J.P.L. Vandewalle, and B.L.R. De Moor, Artificial neuralnetworks for modelling and control of non-linear systems, Kluwer Academic Publishers, Dordrecht, 1996.
[22] J.A.K. Suykens, B. De Moor, and J. Vandewalle, “Robust local stabilityof multilayer recurrent neural networks,” IEEE Trans. Neural Networks,
vol.11, pp.222–229, 2000.
[23] L.L.H. Andrew, “On binary output of cellular neural networks,” Int. J.
Circuit Theory Appl., vol.25, pp.147–149, Mar.–Apr. 1997.
[24] L.O. Chua and C.W. Wu, “ On the universe of stable cellular neural networks,” Int. J. Circuit Theory Appl., vol.20, pp.497–517, Spt.–Oct. 1992.
[25] M. Gilli, “Stability of cellular neural networks and delayed cellular neuralnetworks with nonpositive templates and nonmonotonic output functions,”
IEEE Trans. Circuits Syst. I, vol.41, pp.518–528, Aug. 1994.
[26] N. Takahashi and L.O. Chua, “On the complete stability of nonsymmetriccellular neural networks,” IEEE Trans. Circuits Syst. I, vol.45, pp.754–758,
Jul. 1998.