2007 IEEE Workshop on Nonlinear Circuit Networks
NCN'07, Tokushima, Japan, Dec 7--8, 2007

Clustering of Self-Organizing Map Considering
Winning Frequency
Taku Haraguchi, Masato Tomita, Haruna Matsushita and Yoshifumi Nishio
Department of Electrical and Electronic Engineering,
Tokushima University
Email: taku@ee.tokushima-u.ac.jp, tomita@ee.tokushima-u.ac.jp,
haruna@ee.tokushima-u.ac.jp, nishio@ee.tokushima-u.ac.jp

Abstract— In this study, we propose a new Self-Organizing
Map (SOM) algorithm considering Winning Frequency (calledWF-SOM). The WF-SOM is that each neuron has individuality
by considering winning frequency. We confirm its exact and quick
self-organization in the case of the small number of learning.

where t is the learning step. hc,i (t) is called the neighborhood
function and is described as follows;()
kr i − r c k)2hc,i (t) = α(t) exp −(3),2σ 2 (t)I. I NTRODUCTION

where α(t) is called the learning rate, r i and r c are thevectorial locations on the display grid, and σ(t) corresponds tothe widths of the neighborhood function. Both α(t) and σ(t)decrease monotonically with time as follows;

The Self-Organizing Map (SOM) [1][2] is an unsupervisedneural network introduced by Teuvo Kohonen. SOM has
attracted attention in studies of clustering in recent years.
In this study, we propose a new type of SOM algorithm,
which is called SOM considering Winning Frequency (WFSOM) algorithm. The most important feature of WF-SOM
is that each neuron has individuality by considering winningfrequency. If a neuron becomes a winner less frequency andis nearer to the winner neuron, the neuron is updated moresignificantly. We investigate the behavior of WF-SOM andapply WF-SOM to clustering problems. The efficiencies ofWF-SOM are confirmed by several simulation results.
II. S ELF -O RGANIZING M AP (SOM)
SOM has two-layer structure of the input layer and thecompetitive layer. In the input layer, there are d-dimensional
input vectors xj = (xj1 , xj2 , · · · , xjd ) (j = 1, 2, · · · , M ).
In the competitive layer, m neurons are arranged on the 2dimensional grid. Besides, each neuron has a weight vectors
wi = (wi1 , wi2 , · · · , wid ) (i = 1, 2, · · · , M ) with the same
dimension as the input vector. The range of the input vector
is assumed to be between 0 and 1. The initial values of all theweight vectors are given between 0 and 1 at random.
(SOM1) An input vector xj is inputted to all the neurons atsame time in parallel.
(SOM2) We find the winner neuron by calculating the distances between the input vector xj and the weight vector wiof neuron i. Winner neuron c is the neuron with the weightvector nearest to the input vector xj ;c = arg min{kwi − xj k},i(2)(4)

where T is the maximum number of the learning.
(SOM4) The steps from (SOM1) to (SOM3) are repeated for
all the input data.
III. SOM CONSIDERING W INNING F REQUENCY(WF-SOM)
We explain the learning algorithm of WF-SOM in detail. In
the WF-SOM algorithm, if the winning frequency of a neuron
is smaller and the neuron is nearer to the winner neuron,
the neuron is updated more significantly. Namely, even if it
is seldom a winner, though it is near the winner neuron, it
is updated significantly. Slick neurons always exist. This issimilar to an unreasonable situation at human society. The
initial values of all the weight vectors are given between 0
and 1 at random. The winning frequency of all the neurons
are set to zero : Sc =0.
(WF-SOM1) An input data is inputted to all the neurons at
the same time in parallel.
(WF-SOM2) We find the winner neuron c by calculating thedistance between xj and wi according to Eq. (1).
(WF-SOM3) The winning frequency of winner c is increased
byScnew = Scold + 1.
(5)
(WF-SOM4) The weight vectors of all the neuron are updatedwith considering winning frequency;(1)where k · k is the distance measure, Euclidean distance.
(SOM3) The weight vector of all the neuron are updated aswi (t + 1) = wi (t) + hc,i (t)(xj − wi (t)),α(t) = α(0)(1 − t/T ), σ(t) = σ(0)(1 − t/T ),wi (t + 1) = wi (t) + hc (Sc , t)(xj − wi (t)),
where hc (Sc , t) is a gaussian function described as()(kr c − r i k)2hc (Sc , t) = pc (Sc ) exp − 2,2σW F (Sc , t)

- 29 -

(6)

(7)

1

1

1

0.9

0.9

0.9

0.8

0.8

0.8

0.7

0.7

0.7

0.6

0.6

0.6

0.5

0.5

0.5

0.4

0.4

0.4

0.3

0.3

0.3

0.2

0.2

0.2

0.1

0.1

0

0.1

00

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1(a)
Fig. 1.

00

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

100.1

0.2

0.3

0.4

0.5

0.6(b)

0.7

0.8

0.9

1(c)

Learning Simulation. (a) Input data. (b) Simulation result of conventional SOM. (c) Simulation result of WF-SOM.

where pc (Sc ) is described as(pc (Sc ) = αW F (Sc ) exp −ScM),(8)where α(Sc ) is the learning rate, and σ(Sc , t) corresponds tothe width of the neighborhood function. α(Sc ) decrease with
the number of being winner as follows and σ(Sc , t) decrease
with time and the number of being winner as follows;
Sc,M(9)ScσW F (Sc , t) = σW F (0) ,t
If the winning frequency of a neuron is smaller and the neuron
is nearer to the winner neuron, the neuron is significantlyupdated.
(WF-SOM5) The steps from (WF-SOM1) to (WF-SOM4) arerepeated for all the input data.

quickly than the conventional SOM. However, WF-SOM tends
to produce twists between the neurons.
In order to evaluate the mapping precision of WF-SOM, wedefine a quantization error eq [2].
Quantization Error eq : This measures the average distance
between each input vector and its winner;
1 ∑kxj − w̄j k,N j=1
Neq =αW F (Sc ) = αW F (0)(12)

where w̄j is the weight vector of the corresponding winnerof the input vector xj . Therefore, the small value eq is moredesirable. If the weight vector of the winner neuron is exactly
the same as input data, the value of eq is 0.
The calculated results are summarized in Table 1. We canevaluate the effectiveness of the method using WF-SOM.
Furthermore, the improvement rate is 12.3 [%].
TABLE I

IV. L EARNING S IMULATIONQUANTIZATION ERROR FOR 2- DIMENSIONAL INPUT DATA .

Input data is 2-dimensional random data of 1000 points
whose distribution is non-uniform as Fig.1(a). We repeat thelearning 3 times for all the input data. This is smaller than
the general simulation as the learning times. We consider theconventional SOM and the proposed WF-SOM with M =100
neurons (10 × 10). The parameters of the learning are chosen
as follows;(For SOM)
α(0) = 0.9, σ(0) = 3,(10)αW F (0) = 0.9, σW F (0) = M/3,(11)(For WF-SOM)

eq

Conventional SOM
0.0244

WF-SOM
0.0214

V. C ONCLUSIONSIn this study, we have proposed WF-SOM. We have explained the differences between SOM and WF-SOM withlearning algorithm and have investigated its behavior. Furthermore, we have applied the proposed WF-SOM to clustering
problems in the case of the small number of learning and haveconfirmed its exact and quick self-organization.
In the future, we try to reduce the twists between neurons.
ACKNOWLEDGMENT

Figures 1(b) and (c) show the learning results of the conventional SOM and WF-SOM, respectively. We can see that theconventional SOM does not self-organize all the corners of theinput in the case of the small number of learning. On the otherhand, WF-SOM can self-organize all the corners of the input
although the number of learning is small. From these figures,
we can say that WF-SOM can self-organize more exactly andThis work was partly supported by Yamaha Music Foundation.
R EFERENCES[1] T. Kohonen, “The Self-Organizing Maps,” Neurocomputing, vol. 21,
pp. 1-6, 1998.
[2] T. Kohonen, Self-Organizing Maps, Berlin, Springer, vol. 30, 1995.

- 30 -