2011 International Workshop on Nonlinear Maps and their Applications
NOMA'11, Evora, Portugal, September 15-16, 2011

Hopfield Neural Networks
Operated by Template of Cellular Neural Networks
Yasuhiro Ueda, Yoko Uwate and Yoshifumi Nishio
Dept. Electrical and Electronic Eng.,
Tokushima University
Email: { yasuhiro, uwate, nishio } @ee.tokushima-u.ac.jp

Abstract— In this study, we propose the hybrid system thatconsists of Cellular Neural Networks (CNN) and the Hopfield
Neural Networks (HNN). This system has a part of structure,
characteristics and property in CNN and HNN. The system
has local connectivity property and toroidally-linked boundary
condition. Neurons are updated by the update equation of HNN.
Output is controlled between 1 and -1 by the piece-wise linear
function of CNN. Also the wiring weights are fixed to obtainthe local information like a template in CNN. By the computer
simulations, we investigate the behavior of the proposed system.

I. I NTRODUCTION
Cellular Neural Networks (CNN) [1] were introduced by
Chua and Yang in 1988. The idea of CNN was inspired fromthe architecture of the cellular automata and the neural networks. A different point from the conventional neural networks
is that CNN has local connectivity property. Since the structureof CNN resembles the structure of animals’ retina, CNN canbe used for various image processing applications [2]-[5].
CNN is composed of the basic circuit units called cells. The
cell contains linear and nonlinear circuit elements which are
typically linear capacitors, linear resistors, linear and nonlinear
controlled sources. All cells are connected neighborhood 8
cells.
In addition, Hopfield Neural Networks (HNN) [6] wereintroduced by J. J. Hopfield in 1984. HNN is one of themutual connection neural network and studied chiefly about
the associative memory and optimization problem. A neuron
is connected all other neurons. The neuron updates itself by
the value of other neurons and wiring weights in discretetime. The updated neuron is decided randomly and the wiring
weights are set up depending on the intended purpose. This
network continue to update a neuron and the energy functionof network becomes 0.
In this study, we propose the hybrid system that consistsof CNN and HNN. This system has a part of structure,
characteristics and property in CNN and HNN. In each conventional system, CNN carry out parallel signal processing
in continuous time. CNN has local connectivity property andoutput is controlled between 1 and -1 by the piece-wise linear
function. Also HNN is one of the mutual connection neuralnetworks. The neurons are updated by other neurons and thewiring weights. The wiring weights of the HNN are set up
according to the intended purpose. In the proposed system,
the system has local connectivity property and toroidallylinked boundary condition. Neurons are updated by the update

equation of HNN. Output is controlled between 1 and -1 by thepiece-wise linear function of CNN. Also the wiring weights
are fixed to obtain the local information like a template in
CNN. By computer simulations, we investigate the behaviorof the proposed system. In Sec. II, we explain the basic of thestandard CNN. In Sec. III, we explain the basic of the standard
HNN. In Sec. IV, we describe the structure of the proposedsystem. In Sec. V, simulation results are shown and compare
the conventional CNN with the proposed system. Section VI
concludes this article.
II. C ELLULAR N EURAL N ETWORKSIn this section, we describe the basic structure of CNN.
CNN has M by N processing unit circuits called cells. Cells
are arranged in a reticular pattern to M line by N row. We represent a cell C(i, j) using a variable i which denotes vertical
position and a variable j which denotes horizontal position.
The cell contains linear and nonlinear circuit elements. CNN
is an array of cells. Each cell is connected to its neighborhood
cells according to a template. Usually, the same template isused for all the cells. The boundary condition is changed
according to a kind of the template. CNN has the features oftime continuity, spatial discreteness, nonlinearity and parallel
processing capability.
The state equation and the output equation of the cell C(i, j)
are shown as follows.
State equation:
dvxij
dt

= −vxij +

i+r
∑

j+r
∑

A(i,j;k,l) vykl (t)k=i−r l=j−r

+

i+r
∑

j+r
∑

B(i,j;k,l) vukl (t) + I.

(1)

1(|vxij (t) + 1| − |vxij (t) − 1|).
2

(2)

k=i−r l=j−r

Output equation:vyij (t) =where vx , vy and vu represent a state, an output and an inputof the cell, respectively. In the Eq. (1), A is the feedback
template and B is the control template. These templates andthreshold value I are collectively called the general template.
The performance of CNN decided by the template.

- 62 -

The r-neighborhood of C(i, j) in CNN is defined by
N r(i, j) = {C(k, l) | max { |k − i| , |l − j| } ≤ r,
1 ≤ k ≤ M ; 1 ≤ l ≤ N }.

(3)

where r is a positive integer number.
III. H OPFIELD N EURAL N ETWORKSIn this section, we describe the basic structure of Hopfield
Neural Network (HNN). This neural network is one of themutual connection neural network. A neuron is connected
all other neurons. The neuron updates itself by the value ofother neurons and wiring weights in discrete-time. The wiring
weights are set up depending on the intended purpose. This
network continue to update a neuron and the energy functionof network becomes 0. The energy function of the HNN with
N neurons is defined by the following equation.
Energy function:
E=−

N N
1 ∑∑

2 i=1 j=1

wij xi xj +N∑

θi x i ,

i=1

i̸=j

(6)

IV. H OPFIELD N EURAL N ETWORKS O PERATED BY
T EMPLATE OF C ELLULAR N EURAL N ETWORKSIn this section, we explain the proposed system that consistsof CNN and HNN. (Hereinafter called ”Cellular Hopfield
Networks (CHN)”) CHN has M by N neurons which show
the gray-scale value. The structure of CHN is same with
CNN like Fig. 1(a). One neuron is connected with aroundneighbor neurons. The boundary condition is toroidally-linked
like Fig. 1(b).
The neurons are asynchronously updated due to the following equation.
Update equation:xij (t + 1)

= sgn(

i+r
∑

j+r
∑

(b)

Fig. 1. CHN model. (a) Structure of CHN. (b) Toroidally-linked boundary
condition.

The r-neighborhood of the neuron in CHN is defined by
N r(i, j) = {N euron(k, l) | max { |k − i| , |l − j| } ≤ r,
1 ≤ k ≤ M;

(4)

where wij is the weight between i-th neuron and j-th neuron,and θi is the threshold of the i-th neuron. The weight is 0 wheni = j, because all the units have combined with all the unitsof the others except themselves. The states of the neurons
are asynchronously updated due to the following difference
equation.
Update equation:∑
wij xj (t)),
xi (t + 1) = sgn(
(5)
where sgn is an output function as follows.
Output equation:{
1 (a ≥ 0)
sgn(a) =−1. (a < 0)(a)

T(i,j;k,l) xij,kl (t)), (7)k=i−r l=j−rwhere sgn is an output function which is same with theoutput function of CNN as follows.
Output equation:1sgn(a) = (|a + 1| − |a − 1|).
(8)
2

1 ≤ l ≤ N }, (9)where r is a positive integer number. In our study, we
fix the value of r as 1. Therefore, the size of the templateis fixed as 3 × 3 matrix. Also xij is the state value of theneuron and T(i,j;k,l) is the wiring weights of this network in
the update equation to obtain the local information aroundneighbor neurons.
V. S IMULATION R ESULTS
In this section, we show some simulation results and compare our proposed system with CNN. The results are composedby two tasks. One is a pattern formation and another is theperformance of the “Heat Diffusion” template. The output
image is composed by 2500 pixels(50 × 50 pix.). Therefore
each network is consisting of by 2500 neurons.
A. Pattern formation
In this task, we show the simulation results about a patternformation. The CNN is known to be able to generate periodic
patterns [7]-[9]. The template for pattern formation has designed in CNN [10][11]. We investigate the pattern formationin CHN and compare CHN with CNN. In this task, we apply
following template.

0.15 0 0.15
0.6
0 
T = 0(10)
0.15 0 0.15
Figure 2(a) is an initial image constructed by 1246 black
pixels and 1254 white pixels. Other images in Fig. 2 are
output images. These output images are obtained by the
initial condition and the asynchronously random update of theneuron Figure 2(b) is an output image that having uniform
checkerboard. Figure 2(c) and (d) is uniform black and white
output image, respectively. Figs. 2(b), (c) and (d) are obtainedin our simulation with a high rate. Additionally, we show
distinctive results like Figs. 2(e) and (f). Usually, only one
particular pattern is generated depending on the initial value
and the processing in past study of pattern formation. However,- 63 -

(a)

(b)

(a)

(b)

Fig. 3. Performance of the “Pattern formation” template in CNN. (a) Output
image by substituting the T in the feedback template A. (b) Output image
by substituting the T in the control template B.

B. Performance of the “Heat Diffusion” template

(c)

(e)

In this task, we investigate the performance of CHN for“Heat Diffusion” template by comparing CNN. “Heat Diffusion” template carries out diffusing the input image in CNN.
The boundary condition is fixed as 0 in CNN [7]. This template
gradates the objects to uniform gray scale. The output value
is uniform 0.
“Heat Diffusion” template:
0.1 0.15 0.1
00.15  , B = 0, I = 0.
A =  0.15
(11)
0.1 0.15 0.1

(d)

(f)

Fig. 2. Simulation results of CHN for pattern formation. (a) “Random noise”
image as initial image. (b) Checkerboard as output image. (c) Uniform black
as output image. (d) Uniform White as output image. (e) Checkerboard anda black region are mixed in an output image. (f) Checkerboard and a white
region are mixed in an output image.

two kinds of patterns are coexisted in one output image. Thissystem holds the promise of generating more complicatedpatterns.
Also in CNN, we obtain following output image by T . We
substitute the T in the feedback template A and the control
template B, respectively. For stability of CNN, the centercomponent of the feedback template A is 1 when the T is
substituted in the control template B.
In CNN, the input image is Fig. 2(a). Figures 3(a) and (b)are output image by substituting the T in the feedback template
A and the control template B, respectively. In Fig 3(a), theoutput image is uniform checkerboard. On the other hand,Fig 3(b) is same with input image.
From these simulation results, we show clear distinction of
the performance between CNN and CHN. Also, the generated
patterns are a uniquely-output that coexists two kinds ofpattern.

In the proposed system, the wiring weights T(i,j;k,l) is samewith the feedback template A of “Heat Diffusion” template as
follows.

0.1 0.15 0.1
00.15 
T =  0.15
(12)
0.1 0.15 0.1

(a)

(b)

(c)

Fig. 4. Performance of the “Heat Diffusion” template in CNN and CHN.
(a) “Texture” image as initial image. (b) Output image in CNN. (c) Output
image in CHN.

In Fig. 4(a), the average of the initial image is 0.408. By“Heat Diffusion” template, the output image is uniform 0 likeFig. 4(b). On the other hand, the different result is obtainedby Eq. (12) in CHN. In Fig. 4(c), the output image is uniform
value which is 0.406. We show other simulation results asfollows.
In Fig. 5(a), the average of the initial image is -0.467.
Also the output image is uniform value which is -0.469 like- 64 -

(a)

(b)

(a)

(b)

(c)

(d)

(c)

(d)

Fig. 5.
Other simulation results. (a) “Airplane” image as initial image.
(b) Output image for “Airplane” image in CHN. (c) “Girl” image as initial
image. (d) Output image for “Girl” image in CHN.

Fig. 6. Simulation results by Eq. (13). (a) “Random noise” image as initial
image. (b) Wave form as output image. (c) “Stripe” image as initial image.
(d) Band form as output image.

Fig. 5(b). In Fig. 5(c), the average of the initial image is 0.436.
Also the output image is uniform value which is 0.414 likeFig. 5(d). From these simulation results, we confirm the output
value is similar to the value of the average of initial image. Infact the output value is depended on the initial image.
Additionally, we change the value of template in same size
and investigate its effect. We change the template that is fivetimes as large as Eq. (12) as follows.

confirmed that the output depends on the initial. Also in patternformation, the value of the template and asynchronously
random update influences to the output. As a future work,
we have to investigate the output characteristics with various
templates and images.


0.5 0.75 0.5
00.75 
T =  0.75
0.5 0.75 0.5(13)

For this simulation, the initial image Fig. 6(a) is same with
Fig. 2(a). Figure 6(b) is output image like a wave form. The
width of the waves is changed in each time. Also we obtain another simulation results by another initial image like Fig. 6(c).
In Fig. 6(d), a diagonal band form is generated. All output
images are converged on 1 or -1. Moreover, the convergencetime of Eq. (13) is shorter than that of Eq. (12). From thesesimulation results, the output value, the convergence time andthe generated patterns are depends on the value of template.
Additionally, we consider that the output image has something
to do with the initial image.
VI. C ONCLUSIONSIn this study, we have proposed the hybrid system thatconsists of CNN and HNN. This system has a part of structure, characteristics and property in CNN and HNN. Fromthe simulation results, we have confirmed that this system
output some interesting results. In diffusion template, we have

R EFERENCES[1] L.O. Chua and L. Yang, “Cellular Neural Networks:Theory,” IEEE Trans.
Circuits Syst., vol. 32, pp. 1257-1272, Oct. 1988.
[2] F. Dirk and T. Ronald, “Coding of Binary Image Data using Cellular
Neural Networks and Iterative Annealing,” Proc. of ECCTD’03, vol. 1,
pp. 229-232, Sep. 2003.
[3] M. Namba and Z. Zhang, “Cellular Neural Network for Associative
Memory and Its Application to Braille Image Recognition,” Proc. of
IJCNN’06, pp. 4716-4721, Jul. 2006.
[4] H. Koeppl and L.O. Chua, “An Adaptive Cellular Nonlinear Network andits Application,” Proc. of NOLTA’07, pp. 15-18, Sep. 2007.
[5] T. Kozek, K.R. Crounse, T. Roska and L.O. Chua, “Smart Image Scanning
Algorithms for the CNN Universal Machine,” Proc. of NOLTA’95, vol. 2,
pp. 707-712, 1995.
[6] J. J. Hopfield, “Neurons with Graded Response Have Collective Computational Properties like Those of Two-State Neurons,” Proc. Natl. Acad.
Sci. USA, vol. 81, pp. 3088-3092, 1984.
[7] K. R. Crounse, L.O. Chua, P.Thiran and G.Setti, “Characterization and
Dynamics of Pattern Formation in Cellular Neural Networks, International Journal of Bifurcation and Chaos, vol.6, no.9, pp.1703-1724, 1996.
[8] R. Neubecker, A. Zimmermann, O. Jakoby, “Utilizing nonlinear optical
pattern formation for a simple image-processing task.” Applied Physics
B: Lasers and Optics Volume 76, Number 4, 383-392.
[9] K. R. Crounse and L.O. Chua, “Methods for Image Processing and Pattern
Formation in Cellular Neural Networks: A Tutorial,” IEEE Trans. Circuits
Syst., vol. 42, pp. 583-601, Oct. 1995.
[10] Cellular Sensory Wave Computers Laboratory Computer and Automation Research Institute Hungarian Academy of Sciences, “Cellular Wave
Computing Library (Template, Algorithms, and Programs) Version 2.1”.
[11] Taisuke Nishio and Yoshifumi Nishio, “Periodic Pattern Formation and
Its Applications in Cellular Neural Networks ” IEEE Trans. Circuits Syst.,
vol. 55, pp. 2736-2742, Oct. 2008.

- 65 -