2007 IEEE Workshop on Nonlinear Circuit Networks
NCN'07, Tokushima, Japan, Dec 7--8, 2007

Neural Gas Containing Two Kinds ofUpdate Functions
Keiko Kanda, Haruna Matsushita and Yoshifumi Nishio
Department of Electrical and Electronic Engineering,Tokushima University
Email: kanda@ee.tokushima-u.ac.jp, haruna@ee.tokushima-u.ac.jp, nishio@ee.tokushima-u.ac.jp

Abstract— In this study, we propose a new Neural Gas algorithm which contains two kinds of update functions (called TKFNG). One converges rapidly, and the other converges slowly. In
this method, all neurons learn according to winner’s characterat each learning. We confirm that the proposed Neural Gas
can obtain more effective learning results than the conventional
Neural Gas.

I. I NTRODUCTION
Recently, the Neural Gas network (NG) [1] has attracted
attention as an image-recognition technology. In the learning,
NG uses one type of neurons. However, in the real world, anelephant, which has a slow heart rate, and a mouse, which hasa rapid heart rate, have different senses of time.
In this study, we propose a new Neural Gas algorithm;
Neural Gas containing two kinds of update functions (calledTKF-NG). One converges rapidly, and the other converges
slowly. In this method, all neurons learn according to winner’s
character at each learning. We confirm that TKF-NG canobtain more effective learning results than the conventional
NG.
II. N EURAL G AS C ONTAINING T WO K INDS OFU PDATE F UNCTIONS (TKF-NG)
We explain the learning algorithm of the proposed method,which has two kinds of update functions whose features aredifferent. The network consists of a set A containing M
neurons; A = {c1 , c2 , · · · , cM }. The neurons of the set A areclassified into a set A1 and a set A2 at random. A1 contains m1
neurons, and A2 contains m2 neurons, namely M = m1 +m2 .
The neurons which belong to A1 converges rapidly and theneurons which belong to A2 converges slowly. To be specific,
the convergence rate of the learning rate is different between
A1 and A2 . Each neuron ci (i = 1, 2, · · · , M ) has a ddimensional weight vector wi = (wi1 , wi2 , · · · , wid ). The
initial values of the weight vectors are given at random. The
range of the elements of d-dimensional input data xj =(xj1 , xj2 , · · · , xjd ) (j = 1, 2, · · · , N ) are assumed to be from0 to 1.
(TKF-NG1) An input vector xj is inputted to all the neurons
at the same time in parallel.
(TKF-NG2) Euclid distances dji between the input vector xj
and all the weight vectors wi are calculated;dji = kxj − wi k.

(1)(TKF-NG3) Each neuron ci is assigned a rank ki =0, · · · , (M − 1) depending on their dji . ki denotes the rankof neuron ci . The rank of 0 indicates the neuron closest to xj
and the rank of (M − 1) indicates the neuron farthest fromxj . The closest neuron to xj , namely ki = 0, is defined as awinner.
(TKF-NG4) The weight vectors of the neurons are updated
according towi (t + 1) = wi (t) + EF (t)hλ (t, k)(xj − wi (t)),(2)

where learning rate EF (t) is decided by the winner’s character;( ) TtEf,winner ∈ A1 ,E0

E0

() Tt
EF (t) =(3)p(p + E0 − Ef )
p + E0 − Ef+ Ef − p,winner ∈ A2 ,where t is the time step and T is the total number of training
steps. E0 and Ef are the initial and the final values of E(t),respectively. The fixed parameter p controls the convergence
rate of the learning rate EF (t). In this way, all neurons learn
according to the winner’s character. It is provided p ≥ Ef . Asa result of Eq. (3), the learning rate EF (t) of A1 converges
rapidly. On the other hand, the learning rate EF (t) of A2
decreases with time linearly and converges slowly. In otherwords, if the winner belongs to A1 , all neurons learn basedon the character of A1 , otherwise, all neurons learn basedon the character of A2 . hλ (t, k) is the neighborhood ranking
function described as()kihλ (t, k) = exp −.
(4)λ(t)λ(t) controls the width of the neighborhood function;(λ(t) = λ0λ0λf) Tt,(5)where λ0 and λf are the initial and the final values of λ(t),respectively.
(TKF-NG5) If t < T , return to (TKF-NG1).

- 27 -1

1

1

0.9

0.9

0.9

0.8

0.8

0.8

0.7

0.7

0.7

0.6

0.6

0.6

0.5

0.5

0.5

0.4

0.4

0.4

0.3

0.3

0.3

0.2

0.2

0.2

0.1

0.1

0.1

0

0

0

0.2

0.4

0.6

0.8

1

0

0.2

0.4

(a)

0.6

0.8

01

(b)

00.2

0.4

0.6

0.8

1(c)

Fig. 1. Experimental result of two algorithms for Target data. (a) Input data. (b) Conventional NG. (c) TKF-NG. Points and stars correspond to neurons
belonging to A1 and A2 , respectively.

E0and converges slowly. If the neuron belonging to A1 becomeswinner at time t, all the neuron update with the learning rateEF (t) according to the dashed line in Fig. 2. Conversely, if
the neuron belonging to A2 becomes winner at time t, all theneuron update with the learning rate EF (t) according to thesolid line in Fig. 2. In other words, TKF-NG learns the input
data according to two kinds of features of neuron’s update
functions.
Furthermore, we calculate quantization error Qe [3] toevaluate the learning performance of TKF-NG in comparison
with the conventional NG. This measures the average distance
between each input vector and its winner.

winner ∈ A1
winner ∈ A

0.7

20.6

EF(t)

0.5
0.4
0.3
0.2
0.1
Ef

00500

Fig. 2.

1000

1500

2000
t

2500

3000

3500

TABLE IQ UANTIZATION E RROR Qe OF TWO ALGORITHMS FOR TARGET DATATLearning rate EF (t) according to Eq. 3.

Qe

III. E XPERIMENTAL R ESULTS
We carry out learning simulation for the 2-dimensional input
data, Target data set [2] shown in Fig. 1(a).
Both the conventional NG and TKF-NG have 200 neurons.
In the TKF-NG, the numbers of neurons included in A1 andA2 are equal, namely m1 = m2 = 100. The parameters ofthe learning are chosen as follows; E0 = 0.7, Ef = 0.01,
λ0 = 70, λf = 0.2, p = 50. We use the same E0 , Ef , λ0 , andλf to the conventional NG and TKF-NG for the comparison.
The learning results of the conventional NG and TKF-NG
are shown in Figs. 1(b) and (c), respectively. In Fig. 1(c),we can see that TKF-NG has two kinds of neurons. In thesefigures, we can see that some neurons of TKF-NG are attracted
to the corner input data more strongly than the conventional
one. Thus, TKF-NG can obtain more effective result reflecting
the features of input data than the conventional NG.
Meantime, Fig. 2 shows the learning rate EF (t) according
to Eq. (3) with above parameters when winner belongs to A1(shown as points in Fig. 1(c)) and belongs to A2 (shown asstars in Fig. 1(c)). From this figure, we can see that EF (t)of A1 (shown as a dashed line) converges rapidly, and EF (t)of A2 (shown as a solid line) decreases with time linearly

Conventional NG
0.0162

TKF-NG
0.0128In Table I, the quantization error of TKF-NG is smaller thanthe conventional NG. In this case, the improvement rate fromthe conventional NG to TKF-NG is 21.0%.
IV. C ONCLUSIONSIn this study, we have proposed a new Neural Gas algorithm;
Neural Gas containing two kinds of update functions (TKFNG). We have applied TKF-NG to Target data set and confirmed that TKF-NG can obtain more effective results than theconventional NG in the quantization error. From these results,
we can say that NG containing neurons with plural features of
the update functions can learn the input data more accurately
than NG containing neurons with only one feature.
ACKNOWLEDGMENT
This work was partly supported by Yamaha Music Foundation.
R EFERENCES[1] T. M. Martinetz and K. J. Schulten, “A “neural-gas” network learnstopologies. In T. Kohonen, K. Mäkisara, O. Simula, and J. Kangas,editors,” Artificial Neural Networks, pp. 397–402, 1991.
[2] A. Ultsch, “Clustering with SOM: U*C”, Proc. Workshop on SelfOrganizing Maps, pp. 75–82, 2005.
[3] T. Kohonen, Self-organizing Maps, 2nd ed., Berlin, Springer, 1995.

- 28 -