2009 IEEE Workshop on
Nonlinear Circuit Networks, Tokushima, Dec 11-12, 2009

CNN Template Learning To Obtain Desired Images
Using Back Propagation Algorithm
Masashi Nakagawa, Takashi Inoue and Yoshifumi Nishio
Department of Electrical and Electronic Engineering, Tokushima University
2-1 Minami-Josanjima, Tokushima, 770-8506, Japan
Email: {miyabi, takashi, nishio}@ee.tokushima-u.ac.jp
I. I NTRODUCTION
Cellular neural networks (CNN) were introduced by Chuaand Yang in 1998 [1]. The idea of the CNN was inspired
from the architecture of the cellular automata and the neuralnetworks. Unlike the conventional neural networks, the CNN
has local connectivity property. Since the structure of the CNN
resembles the structure of animals retina, the CNN can be usedfor various image processing applications [2]-[4].
Wiring weights of the cells of the CNN are established byparameters called the template. The template is most importantparameter, because performance of the CNN is decided by thetemplate. Thus, some template design methods as template
learning using GA algorithm are proposed. These works areimportant subject in the studies of the CNN [5].
In this paper, template learning of cellular neural networks
using back propagation algorithm is proposed. In our proposed
CNN, we build back propagation algorithm into the feedback
part of CNN. Back propagation algorithm is inspired formback propagation neural networks [6]. Back propagation neuralnetworks operates with a feed forward neural network whichis composed of an input layer, a hidden layer and an output
layer, and the effectiveness of the back propagation algorithm
has been confirmed in learning performance. In this paper, thetemplate of CNN is dynamically updated in each step using theback propagation algorithm. Computer simulations using onedimensional image show that the back propagation algorithm iseffective for template learning of CNN. At the moment, we donot say that the proposed template learning method exhibited asuperior performance than the other template learning method.
However, we feel that we obtained some results to broaden theresearch on the template learning of CNN.

A. Processing Procedure for Learning
Processing procedure for learning is explained as follows.
Step 1.An input image and a desired image (a teacherimage) are prepared. An initial template is decided
at random. Because we use random in order to beable to learn the template for any initial values.
However, in our learning method, we should mention
the stability of template in CNN. Because the valueof an output may become unstable with the valueof a template. The condition of stability is that thecenter element of template A maintains 1 and thesymmetry of each element.

II. P ROPOSED T EMPLATE L EARNING M ETHOD
In this section, we describe the proposed template learningmethod using back propagation algorithm. Our template learning method is inspired from perception of the back propagation
algorithm. In particular, in the back propagation algorithm,
parameter is changed by using error between an output and adesired value, in order to bring an output close to a desiredvalue. From this perception of learning, template of CNN
is changed by using error between an output image and adesired image. Additionally, this template learning carry out
dynamically. Namely, processing by the CNN and template
learning are carried out at the same time.

- 93 -Step 2.An input image is processed by CNN using RungeKutta method. At this time, unit width of RungeKutta method is made small, and only a few processes. It is for learning dynamically not using theprocessed image but using the present image currently processed. At this time, the value of V x before
processing is defined as V xpast , and the value ofV x after processing is defined as V xnow . It is for
deciding the direction of correction by comparing
with the value of V x.
Step 3.The error between an output image and a desired
image is calculated for every pixel of an output anda desired image. An absolute value is calculated inorder to make the error of each pixel into a positivevalue.
An error Ep is calculated as the following equation:√Ep = (tp − op )2 ,(1)where p is the number of the pixel, tp denotes thevalue of the desired data of the pth pixel, and op
denotes the value of the output data of the pth pixel.
Step 4.The corrected direction δp is decided using error Ep
as the following equation:δp = Ep × (V xnow
− V xpast)pp(2)Step 5.The correction quantity ∆p for every pixel is calculated as the following equation:∆p = η × δp × V xnow,p(3)where η is a proportionality factor known as thelearning rate.
Step 6.The errors of all the pixels are added.
D=p∑III. S IMULATIONIn this section, we show the learning results using theproposed learning method.
A. Input Image and Desired Image

∆i(4)

i=1

We prepare a one-dimensional input image and desired
image as Figs. 2 and 3.

The calculation adds all the values of a template.
However, it does not include the center element of
the template A, B, and the bias I. That is, a templateis changed as the following equation.
ARenewed
10ore= ABef+ D,
10

ARenewed
12ore= ABef+ D,
12Renewed
B10Renewed
B12Bef ore
B10+ D,Bef ore
B12+D==Input image:Fig. 2.

Input image

Fig. 3.

Desired image

Desired image:(5)

The reason of the above setting is that the stabilityof the template can be kept.
Step 7.Step.2 - Step.6 are repeated until the process converges.
Thus, by updating a template, an output image is broughtclose to a desired image.
Finally, the diagram showing the flow of the process isshown in Fig. 1.

The black pixels correspond to the value +1, while the white
pixels correspond to the value -1.
A teacher image is made from the following two rules.
Rule 1. If the colors of the neighbors of a certain cell is thesame color, the cell’s color turns into its neighboringcolor.
Rule 2. If the colors of the neighbors of a certain cell isdifferent, the cell’s color does not change.
That is to say, this rule is like “Reversi”.
B. Parameters for Learning
The following parameters are used in this simulation.
TABLE IPARAMETERS
Name of parameter
Step size
Maximum iteration number
Number of pixels
Number of learning loops
Learning rate

value
0.01
1030
100
0.01

C. Simulation Results
As a result of carrying out computer simulations, thefollowing images and templates are obtained. In our proposed
learning method, because elements of the initial template areset at random. Therefore, the results are always different. Theresults shown below are some examples of obtained results.
Learning result 1:
Figure 4 shows the example of the output image using theproposed learning method.

Fig. 4.

Fig. 1.

Procedure for learning

Result image

In Fig. 4, we can see that the three pixels differ fromthe desired image. In this case, we obtained the template asfollows.

- 94 -obtained. Therefore, we want to obtain a desired image byraising accuracy.
In the future, we want to learn the template which can carryout processing two-dimensional images, and processing grayscale images using back propagation algorithm.

Obtained template:
000A =  0.917804 1 0.917804  ,
000
000
B =  1.538764 0.113712 0.644816  ,000
I=0R EFERENCES(6)

In obtained template (6), elements of feedback template Astability.
Learning result 2:
The following results are obtained by using the same
parameters as the learning result 1.

Fig. 5.

[1] L.O. Chua and L. Yang, “Cellular Neural Networks:Theory,” IEEE Trans.
Circuits Syst., vol. 32, pp. 1257-1272, Oct. 1988.
[2] F. Dirk and T. Ronald, “Coding of Binary Image Data using Cellular
Neural Networks and Iterative Annealing,” Prof. of ECCLES, 03, vol. 1,pp. 229-232, Se. 2003.
[3] M. Samba and Z. Hang, “Cellular Neural Network for Associative
Memory and Its Application to Braille Image Recognition,” Prof. of 06,
pp. 4716-4721, Jul. 2006.
[4] H. Koeppl and L.O. Chua, “An Adaptive Cellular Nonlinear Network andits Application,” Proc. of NOLTA’ 07, pp. 15-18, Sep. 2007.
[5] T. Kozek, T. Roska, and L.O. Chua, “Genetic Algorithm for CNN
Template Learning,” IEEE Trans. Circuits Syst., vol.40, pp. 392-401, Jun.
1993.
[6] D.E. Rumelhart, G.E. Hinton and R.J. Williams, “Learning Internal
Representations by Error Propagation,” Parallel Distributed Processing,vol. 1, MIT Press, MA, pp. 318-362, 1986.

Result image

In Fig. 5, the same result as the desired image can obtained.
The different template outputting the desired image are obtained by the proposed learning method.
Obtained template:
000
A =  0.712821 1 0.712821  ,000000B =  1.425642 0.7734 1.911821  ,000
I=0(7)

Although the learning parameters are the same, different
templates are obtained. Because the initial values of thetemplates were decided at random.
The “Reversi” was able to be performed when processing other input images by the conventional CNN using theobtained templates as template (7). However, since thesetemplates are updated dynamically, it is not a template in
the conventional CNN which can certainly realize “Reversi”.
Additionally, we should mention that we obtained the template performing “Reversi” only for working one-dimensional
images in this study.
IV. C ONCLUSIONSIn this paper, we have proposed a template learning methodof cellular neural networks (CNN) using the back propagation
algorithm. The template was learned by using the error between a desired image and an output image. Moreover, it wasable to bring close to the desired image, changing a templateby processing dynamically using back propagation algorithm.
However, since this proposed method has the characteristicof processing dynamically, a desired image is not necessarily- 95 -