WCCI 2010 IEEE World Congress on Computational Intelligence
July, 18-23, 2010 - CCIB, Barcelona, SpainIJCNNSpace-Varying Cellular Neural Networks
Designed by Hopfield Neural Network
Yasuhiro Ueda, Masakazu Kawahara, Takashi Inoue, Yoko Uwate and Yoshifumi Nishio

Abstract— In this study, we propose a space-varying cellularneural network (CNN) designed by Hopfield neural network(Hopfield NN). CNN is classified into two types of system
like space-invariant system and space-varying system. Spaceinvariant means that all cells have identical template. On theother hand, space-varying means that all cells do not haveidentical template according to the state values of the cell andneighbor cells and so on.
The proposed CNN is the space-varying system and it isdesigned by using an associative memory ability of Hopfield
NN. In general, the design of space-varying systems is noteasy. However, we can set one of prepared existing templateson each cell of CNN according to the retrieved pattern by
Hopfield NN to which some typical local image structures areembedded. Namely, we need only some existing templates andtheir associated patterns. Some simulation results show the basic
properties of the proposed CNN and its effectiveness.

I. I NTRODUCTIONELLULAR Neural Networks (CNN) [1] were introduced by Chua and Yang in 1988. The idea of CNN was
inspired from the architecture of the cellular automata andthe neural networks. A different point from the conventional
neural networks is that CNN has local connectivity property.
Since the structure of CNN resembles the structure of animals’ retina, CNN can be used for various image processing
applications [2]-[5]. CNN is composed of the basic circuit
units called cells. The cell contains linear and nonlinear
circuit elements which are typically linear capacitors, linear
resistors, linear and nonlinear controlled sources.
Wiring weights of the cells are established by parameters
called the template. The performance of the CNN is decided
by the template. If the templates of all the cells in the CNN
are identical, the system is called space-invariant, while if thetemplates of all the cells in the CNN are not identical, thesystem is called space-varying. Typically, the space-invariant
system is mainly used in the studies of the CNN because theimplementation cost is low. However, the performance of thespace-invariant system may be limited because the essential
processing dynamics are identical for every position. On theother hand, the performance of space-varying system can bebetter although the design of the template and implementation are much more difficult. Therefore, the research of thespace-varying system is an interesting subject for developing
a new direction of CNN. In the research of the space-varying
system, the design method of CNN is important problem andC

it influences the performance of CNN. Sumitomo and Ushidaproposed a design method using Fuzzy Inference [6]. They
could design the space-varying CNN that takes advantage of
Fuzzy inference and have confirmed the effectiveness of thespace-varying CNN.
In this study, we propose a space-varying CNN designed
by using an ability of the associative memory of Hopfieldneural network (Hopfield NN) [7]. In the design method,
firstly, we prepare some two-dimensional patterns with thesize of 5×5 and some existing templates are associated withthese patterns as one to one. Secondly, the two-dimensional
patterns are embedded to the Hopfield NN with the same
size. Thirdly, we input each pixel and its two-neighborhood
pixels (namely 5×5 pixels) of an input image into Hopfield
NN. Hopfield NN retrieves one of the embedded patterns
which is the most similar to the pattern of each pixel and its
two-neighborhood pixels. Fourthly, according to the retrieved
pattern, its associated template is set on the cell of CNN.
Namely, all the templates of the space-varying CNN are
decided. Finally, the input image is processed by using thespace-varying CNN.
In Sec. 2, we review the standard CNN. In Sec. 3, we
review the standard Hopfield NN. In Sec. 4, we describe thestructure of the space-varying CNN designed by Hopfield
NN. In Sec. 5, simulation results of some image processing
tasks are shown. Section 6 concludes the article.
II. C ELLULAR N EURAL N ETWORKS [1]
CNN has M by N processing unit circuits called cells.
Cells are arranged in a two-dimensional lattice pattern to
M lines by N rows. We represent a cell C(i, j) using avariable i which denotes vertical position and a variable jwhich denotes horizontal position. Each cell is connected toits neighborhood cells according to a template. Usually, thesame template is used for all the cells except for boundary
cells. CNN has the features of time continuity, spatial discreteness, nonlinearity and parallel processing capability.
The state equation and the output equation of the cellC(i, j) are shown as follows.
State equation:

Yasuhiro Ueda, Masakazu Kawahara, Takashi Inoue, Yoko Uwate andYoshifumi Nishio are with the Department of Electrical and Electronic
Engineering, Tokushima University, 2-1 Minami-Josanjima, Tokushima 7708506, Japan (Phone: +81 88 656 7470; e-mail: {yasuhiro, kawahara, uwate,nishio}@ee.tokushima-u.ac.jp).

978-1-4244-8126-2/10/$26.00 c 2010 IEEE

3770

dvxij
dt

= −vxij +

i+r


j+r


A(i,j;k,l) vykl (t)k=i−r l=j−r

+

i+r


j+r


k=i−r l=j−r

B(i,j;k,l) vukl (t) + I.

(1)

x1(t)

Output equation:
1
vyij (t) = (|vxij (t) + 1| − |vxij (t) − 1|)
2

(2)

w2

x2(t)

・
・
・
・

where vx , vy and vu represent a state, an output and an inputof the cell, respectively. In Eq. (1), A is the feedback templateand B is the control template. These templates and thresholdvalue I are collectively called the general template.
The r-neighborhood of C(i, j) in CNN is defined by
1 ≤ l ≤ N }.

wn

(3)

θ

where r is a positive integer number. In our study, we fix thevalue of r as 1.

Fig. 1.

III. H OPFIELD N EURAL N ETWORK W ORKING AS
A SSOCIATIVE M EMORY
Associative memory is a system which retrieves an embedded pattern or its reversed pattern being similar to an inputted
pattern. Noisy patterns can be associated or distorted patterns
can be recognized by a well-constructed associative memory.
The Hopfield NN is used as an associative memory by
exploiting the property that the network has multiple stablestates. Namely, if the parameters of the network can bedecided in such a way that the patterns to be stored become
stable states of the network, the network produces a stored
pattern that is similar to an input pattern. The energy functionof the Hopfield NN with N neurons and P stored binary
patterns is defined by the following equation.
N N
N

1 
wij xi xj +θi xi ,
E=−
2 i=1 j=1
i=1

f

x(t＋１)xn(t)N r(i, j) = {C(k, l) | max { |k − i| , |l − j| } ≤ r,
1 ≤ k ≤ M;

w1

Neuron model of conventional Hopfield NN model.

IV. S PACE - VARYING CELLULAR NEURAL NETWORKSDESIGNED BY H OPFIELD NEURAL NETWORKIn this section, we describe the space-varying CNN by
Hopfield NN. The proposed CNN is a space-varying system,
and it is designed by using the ability of the associative
memory of Hopfield NN. In general, a design of the spacevarying system needs to solve complicated differential equations. However, in the proposed design method, we do nothave to do it.
Input image

Hopfield
NN(4)
RELATED

where wij is the weight between i-th neuron and j-th neuron,and θi is the threshold of the i-th neuron. The weight wij isgiven as follows.
 P(p) (p)xj(i = j)p=1 xiwij =(5)0 (i = j).
The weight wij is 0 when i = j, because all the units havecombined with all the others except themselves. The states of
the neurons are asynchronously updated due to the following
difference equation:
wij xj (t)),(6)
xi (t + 1) = sgn(

Template1

Template2

Template3

Space-Varying Cellular Neural Networks
Fig. 2.

Design method by Hopfield NN.

Figure 2 shows the design method of the space-varying
CNN by Hopfield NN. Each step of the design method is
explained as follows.

(7)

Step 1: We prepare some two-dimensional patterns forthe memory of Hopfield NN and some existing templates
associated with these patterns, respectively. By using existing
templates, we are able to shorten the time of design. Exampleof two dimensional patterns and existing templates are shownin Fig. 3.

Figure 1 shows the neuron model of the conventional
Hopfield NN model.

Step 2: Hopfield NN memorizes the two-dimensional patterns 1, 2 and 3 as Fig. 3 by using Eq. (5). In this article,

i=jwhere sgn is an output function as follows:
1 (a ≥ 0)
sgn(a) =−1 (a < 0).

3771

Hopfield NN

Template 1
Edge Detection

AssociateddPattern 1

Pattern 2

Input patterns

Associated pattern

Template 2
Heat Diffusion

Cij

AccordTemplate 3
Logic Not

T
Templateloff C ij

Pattern 3
Fig. 3.

Template 2 Associated

Example of patterns associated with templates.

Memorized pattern
the two-dimensional patterns are only binary images.
Step 3: Figure 4 shows the template decision procedureby the associated patterns. In the example of Fig. 4, after
the input pattern is inputted, some two-dimensional patterns
and pattern of C(i, j) and its two-neighborhood pixels arecompared. Hopfield NN retrieves one of the embedded twodimensional patterns which is similar to the pattern of C(i, j)
and its two-neighborhood pixels. In this example, “Pattern 2”
in Fig. 3 is retrieved from the inputted pattern. Namely, thelocal feature of the input image is classified into one of theembedded images.

Fig. 4.

Template decision procedure by associated patterns.

through a series of steps as Fig. 2.
Hence, the input image is processed using each suitable
template of the space-varying CNN designed by HopfieldNN.
V. S IMULATION RESULTS

Step 4: From the retrieved pattern in Step 3, template of cellwhich is represented C(i, j) is decided the “Heat Diffusion”
template in Fig. 3. Namely, the state equation and output
equation of C(i, j) are shown as follows.

In this section, we show some simulation results using
the designed space-varying CNN. All existing templates arefound in [8].

State equation:

In this task, we prepare three patterns as Fig. 5 and three
existing templates as Eqs. (10), (11) and (12).
The three patterns show the three directions like vertical,
horizontal and diagonal line, respectively. The three existing
templates have completely different performance of the image processing. If Hopfield NN retrieves the pattern 1, 2 or
3 in the certain pixel of an input image, a cell corresponding
to the pixel has “White Filler” template, “Heat Diffusion”
template or “Logic Not” template, respectively.
Existing templates associated with patterns in Fig. 5:
Pattern 1 : “White Filler” template:
⎡
⎤
0 0 0
A = ⎣ 0 2 0 ⎦ , B = 0, I = −4.
(10)
0 0 0

dvxij
dt

= −vxij +

i+r


j+r


A2(i,j;k,l) vykl (t)k=i−r l=j−r

+

i+r


j+r


B2(i,j;k,l) vukl (t) + I2 . (8)k=i−r l=j−r

Output equation:
1(|vxij (t) + 1| − |vxij (t) − 1|).
(9)
2
In Eq. (8), A2 , B2 and I2 represent the “Heat Diffusion”
template in Fig. 3. Additionally, the output equation is thesame as the conventional CNN.
vyij (t) =A. Task 1: Texture segmentation

Step 5: Hopfield NN carries out the Steps 3 and 4 for the
all cells of CNN. Finally, the space-varying CNN is designed

3772

This template makes image white all around.

be segregated because each texture region having different
template converges to different value. Namely, we consider
that each image processing is carried out at the same timein each texture region by the designed space-varying CNN.
B. Task 2: Multiple direction lines detector with noise removal(a) Pattern 1.
Fig. 5.

(b) Pattern 2.

(c) Pattern 3.

Three patterns for the Task 1.

Pattern 2 : “Heat Diffusion” template:⎡⎤
0.1 0.15 0.1
00.15 ⎦ , B = 0, I = −1.
A = ⎣ 0.15
0.1 0.15 0.1(11)This template gradates the objects to uniform gray scale.
Pattern 3 : “Logic Not” template:
⎡
⎤
⎡
0 0 0
0 0
A = ⎣ 0 1 0 ⎦ , B = ⎣ 0 −2
0 0 0
0 0

In this task, we prepare three patterns as Fig. 7 and two
existing templates as Eqs. (13) and (14). The three patterns
show the horizontal line, vertical line and noises pattern,
respectively. The two existing templates have completely
different performance of image processing. If Hopfield NN
retrieves the pattern 1 or 2 in the certain pixel of an input
image, a cell corresponding to the pixel has “Edge Detection”
template. Similarly, if Hopfield NN retrieves the pattern 3 inthe certain pixel of an input image, a cell corresponding to
the pixel has “Small Object Remover” template.

⎤00 ⎦ , I = 0. (12)0

In binary images, this template turns over white and black.
(a) Pattern 1.

(b) Pattern 2.

Fig. 7.

(c) Pattern 3.

Three patterns for Task 2.

Existing templates associated with patterns in Fig. 7:
Patterns 1 and 2 : “Edge Detection” template:
⎡

0
A=⎣ 0
0

(a)

0
1
0

⎤
⎡
0
−1
0 ⎦ , B = ⎣ −1
0
−1

−1
8
−1

⎤
−1
−1 ⎦ , I = −1.
−1(13)This template can detect the edge of objects.

(b)Fig. 6. Texture segmentation result using the designed space-varying CNN.
(a) Texture input image. (b) Texture segmentation result for Fig. 6(a).

Figure 6 shows the texture segmentation result using thespace-varying CNN designed by Hopfield NN based on threepatterns as Fig. 5 and three existing templates Eqs. (10), (11)and (12). Figure 6(a) shows the texture input image having
three types of texture corresponding vertical, horizontal anddiagonal line, respectively. From Fig. 6(b), we can say thatthe cells of each texture region having vertical, horizontal
and diagonal line are associated with the patterns 1, 2 and 3,
respectively. Since “White Filler” template is associated withthe vertical lines, the region with vertical lines converges towhite. Also, since “Heat Diffusion” template is associatedwith the horizontal lines, the region with horizontal lines
is converged on uniform gray scale. Additionally, since“Logic Not” template is associated with the diagonal lines,
the region with diagonal lines becomes reversal of whiteand black. In Fig. 6(b), we can see that each texture canPattern 3 : “Small Object Remover” template:
⎡
⎤
1 1 1
A = ⎣ 1 2 1 ⎦ , B = 0, I = 0.
1 1 1

(14)

This template removes the small objects.
Figure 8 shows the multiple direction lines detector withnoise removal result using the space-varying CNN designed
by Hopfield NN based on the three patterns in Fig. 7 and thetwo templates Eqs. (13) and (14).
Figure 8(a) shows the binary input image having a circle
and many noises. From Fig. 8(b), we can say that Hopfield
NN retrieves the pattern 3 in the pixels of noise. Additionally,
Hopfield NN retrieves the pattern 1 or 2 in the pixels ofmultiple direction lines which are similar to the vertical or
horizontal line. Therefore, multiple direction lines which aresimilar to the vertical or horizontal line can be detected
although diagonal line can not be detected. In this simulation,

3773

(a)

(b)

(a)

(b)

Fig. 8. Multiple direction lines detector with noise removal result using
the designed space-varying CNN. (a) Input image. (b) Some directions line
detector with noise removal result for Fig.8(a).

Fig. 10. Reversed edge detection results using the designed space-varying
CNN. (a) Binary input image. (b) Reversed edge detection results for
Fig.10(a).

noises of the input image can be removed at the same timeby the effect of “Small Object Remover” template. Namely,
we consider that two different image processings are carriedout at the same time by the designed space-varying CNN.

Figure 10 shows the reversed edge detection result using
the space-varying CNN designed by Hopfield NN based on
the three patterns in Fig. 9 and the two existing templates
Eqs. (12) and (15). Figure 10(a) shows the binary input
image. From Fig. 10(b), by setting the three patterns ofFig. 9, cells corresponding to the periphery of edges of
input image are associated with the patterns 1 or 2, and has“Logic Not” template. On the other hand, other cells areassociated with the pattern 3, and have “Black Propagation”
template. By the “Logic Not” template, the color of the edgeof alphabet and its circumference turns over. In fact, theedge of alphabet in input image changes to white, and its
circumference changes to black. The other cells turn blackby effect of “Black Propagation” template. As a result, whiteedge equals in the original edge. Namely, we consider thatthe each image processing is carried out at the same timein the circumference of the edges and other region by thedesigned space-varying CNN.

C. Task 3: Reversed edge detection
In this task, we prepare three patterns as Fig. 9 and two
templates as Eqs. (13) and (15). The three patterns show
the vertical edge, horizontal edge and background region,
respectively. The two existing templates have completely
different performance of image processing. If Hopfield NN
retrieves the pattern 1 or 2 in the certain pixel of an input
image, a cell corresponding to the pixel has “Logic Not”
template. Similarly, if Hopfield NN retrieves the pattern 3 inthe certain pixel of a input image, a cell corresponding to
the pixel has “Black Propagation” template.

(a) Pattern 1.
Fig. 9.

(b) Pattern 2.

VI. C ONCLUSIONSIn this study, we have proposed a space-varying CNN
designed by using ability of associative memory of Hopfield
NN. Some simulation results showed the basic propertiesof the proposed CNN and its effectiveness. According toour simulation, we consider that the setting of the preparedpatterns is the most important for the performance of image processing using the proposed space-varying CNN. In
other words, the proposed CNN can perform various image
processing by using different patterns and templates. The
proposed space-varying CNN has a possibility that a new
direction of CNN is developed. In the future work, we would
like to carry out in gray-scale images and more complicatedimage processing.

(c) Pattern 3.

Three patterns for Task 3.

Existing templates associated with patterns in Fig. 7:
Patterns 1 and 2 : “Edge Detection” template:
Existing templates associated with patterns in Fig. 9:
Pattern 1 and 2 : “Logic Not” template as Eq. (12):Pattern 3 : “Black Propagation” template:⎡⎤
0.25 0.25 0.25
3
0.25 ⎦ , B = 0, I = 3.75.
A = ⎣ 0.25
0.25 0.25 0.25

R EFERENCES(15)

This template propagates the inputted black.

3774[1] L.O. Chua and L. Yang, “Cellular Neural Networks:Theory,” IEEE
Trans. Circuits Syst., vol. 32, pp. 1257-1272, Oct. 1988.
[2] F. Dirk and T. Ronald, “Coding of Binary Image Data using Cellular
Neural Networks and Iterative Annealing,” Proc. of ECCTD’03, vol. 1,
pp. 229-232, Sep. 2003.
[3] M. Namba and Z. Zhang, “Cellular Neural Network for Associative
Memory and its Application to Braille Image Recognition,” Proc. of
IJCNN’06, pp. 4716-4721, Jul. 2006.

[4] H. Koeppl and L.O. Chua, “An Adaptive Cellular Nonlinear Network
and its Application,” Proc. of NOLTA’07, pp. 15-18, Sep. 2007.
[5] T. Kozek, K.R. Crounse, T. Roska and L.O. Chua, “Smart Image
Scanning Algorithms for the CNN Universal Machine,” Proc. of
NOLTA’95, vol. 2, pp. 707-712, 1995.
[6] K. Sumitomo and A. Ushida, “A Design Method of Cellular Neural Networks Using Fuzzy Inference,” IEICE Technical Report,
no. NLP98-443, pp. 39-46, Dec. 1998.
[7] J.J. Hopfield, “Neurons with Graded Response Have Collective Computational Properties Like Those of Two-State Neurons,” Proc. Natl.
Acad. Sci. USA, vol. 81, pp. 3088-3092, 1984.
[8] Cellular Sensory Wave Computers Laboratory Computer and Automation Research Institute Hungarian Academy of Sciences, “Cellular
Wave Computing Library (Template, Algorithms, and Programs) Version 2.1”.

3775