社団法人 電子情報通信学会
THE INSTITUTE OF ELECTRONICS,
INFORMATION AND COMMUNICATION ENGINEERS

信学技報
TECHNICAL REPORT OF IEICE.

Lazy Self-Organizing Map for Eﬀective Self-Organization
Taku HARAGUCHI† , Haruna MATSUSHITA† , and Yoshifumi NISHIO†† Department of Electrical and Electronic Engineering, Tokushima University
2–1 Minami-Josanjima, Tokushima 770–8506, Japan
E-mail: †{taku,haruna,nishio}@ee.tokushima-u.ac.jp
Abstract The Self-Organizing Map (SOM) is a famous algorithm for the unsupervised learning and visualization
introduced by Teuvo Kohonen. This study proposes the Lazy Self-Organizing Map (LSOM) algorithm which reﬂects
the world of worker ants. In LSOM, three kinds of neurons exist: worker neurons, lazy neurons and indecisive neurons. We apply LSOM to various input data set and conﬁrm that LSOM can obtain a more eﬀective map reﬂecting
the distribution state of the input data than the conventional SOM.
Key words self-organizing maps (SOM), clustering, data mining

1. Introduction
Clustering is one of typical analysis techniques and is studied for many applications, such as statement, pattern recognition, data mining. In recent years, the Self-Organizing Map(SOM) [1] has attracted attention for the study on clustering [2]. SOM is an unsupervised neural network introduced
by Kohonen in 1982 and is a simpliﬁed model of the selforganization process of the brain. SOM can classify input
data according to similarities, which are obtained by the distance between neurons, and is applied to wide ﬁelds of dataclassiﬁcations. In the learning algorithm of SOM, a winner
neuron and its neighboring neurons are updated. A learning rate, which denotes the updating degree of neurons, decreases with time, and the neuron nearer to the winner issigniﬁcantly updated. This means that all the neurons canbe updated, in so far as it is near to the input data or thewinner neuron. In other words, all the neurons of the conventional SOM are good worker neurons.
Meanwhile, it is believed that God does not create an unnecessary thing even if it seems to useless. There is a report that 20 percent of worker ants are “lazy” (as Fig. 1) [3].
These ants keep still or stay around their nests. However, theresearchers think that the lazy ants have certain role. Because, in another experiment, the ants group, which includes
the lazy ants at food collections, can collect more foods than
the group which includes only the worker ants. The worker
ants can collect food eﬃciently, however, it may be hard toﬁnd new foods just because they consider the eﬃciency. In
addition, it is thought that chances of ﬁnding foods increase
because the lazy ants maunder without working. In otherwords, we can obtain better results in the case that uselessthings exist than only excellent things.
In this study, we propose a new type of SOM algorithm,
which is called Lazy SOM (LSOM) algorithm. The important feature of LSOM is that three kinds of neurons exist; worker neurons, lazy neurons, which do not work andWorker ants

Lazy ants

Fig. 1 World of worker ants. Some worker ants are “lazy”.

indecisive neurons which are the neighborhoods of the lazyneurons. The learning rate of the lazy neurons is smaller
than the ones of the worker neurons. The learning rate of
the indecisive neurons become small due to the lazy neurons. We can say that LSOM carries out learning reﬂecting
the world of worker ants.
We explain the learning algorithm of LSOM in detail in
Section 3.. In Section 4., we apply LSOM to 2-dimensionaland 3-dimensional input data, which have some clustering
problems. Furthermore, we explain the learning behaviorsof LSOM in detail. Learning performances are evaluatedboth visually and quantitatively using two measurements
and are compared with the conventional SOM. We conﬁrm
that LSOM can obtain a more eﬀective map reﬂecting the
distribution state of the input data than the conventional
SOM.

2. Self-Organizing Map (SOM)
SOM has a two-layer structure of the input layer and thecompetitive layer. In the input layer, there are d-dimensional—1—input vectors xj = (xj1 , xj2 , · · · , xjd ) (j = 1, 2, · · · , N ). In
the competitive layer, M neurons are arranged on the 2dimensional grid. Each neuron has a weight vectors wi =(wi1 , wi2 , · · · , wid ) (i = 1, 2, · · · , M ) with the same dimension as the input vector. The range of the input vector isassumed to be between 0 and 1. The initial values of all theweight vectors are given between 0 and 1 at random.
(SOM1) We input an input vector xj to all the neurons atsame time in parallel.
(SOM2) We ﬁnd a winner neuron by calculating the distances between the input vector xj and the weight vector
wi of neuron i. The winner neuron c is the neuron with theweight vector nearest to the input vector xj ;c = arg min{kwi − xj k},(1)iwhere k · k is the distance measure, in this study, we use Euclidean distance.
(SOM3) The weight vector of all the neurons are updatedaswi (t + 1) = wi (t) + hc,i (t)(xj − wi (t)),

4
2

4

1

1

Worker neuron

Indecisive neuron

2

1

2

1

Lazy neuron

42

4
Indecisive neurons increase
with time !!

Fig. 2 LSOM contains three kinds of neurons: worker neuron,lazy neuron and indecisive neuron. Each number denotes
neighborhood distance between lazy neuron and each neuron.

Start

p ∈ Slazy(2)

where t is the learning step. hc,i (t) is called the neighborhood function and is described as(

kr i − r c k2
hc,i (t) = α(t) exp −2σ 2 (t)

)

Input

,

(3)
Find winner

where r i and r c are the vectorial locations on the display
grid, α(t) is called the learning rate, and σ(t) corresponds
to the widths of the neighborhood function. Both α(t) andσ(t) decrease monotonically with time, in this study, we use
the following;(α(t) = α(0) 1 −

t
T

)

(

, σ(t) = σ(0) 1 −t
Tc ∈ S lazy
Yes

Find f andChange c for f

)

No

,

(4)

where T is the maximum number of the learning.
(SOM4) The steps from (SOM1) to (SOM3) are repeated
for all the input data.

D ≥1
Yes
Find N lazyj = j +1

Choose α

3. Lazy Self-Organizing Map (LSOM)In this study, we propose Lazy SOM (LSOM). The important feature of LSOM is that three kinds of neurons exist (asFig. 2): worker neurons, lazy neurons, which do not work,
and indecisive neurons which are neighborhoods of the lazyneuron. The updating degree of the lazy neurons is smaller
than the ones of the worker neurons. Furthermore, the updating degrees of the lazy neurons’ neighbors become small
due to the lazy neuron, hence the name is “indecisive”. The
lazy neuron becomes the worker neuron whenever it becomes
the winner, and another neuron becomes lazy. The numberof the indecisive neuron increases with time.
3. 1 Learning Algorithm
We explain the learning algorithm of LSOM in detail. A
ﬂowchart of the learning algorithm is shown in Fig. 3. In
LSOM, M neurons are arranged as a regular 2-dimensional
grid. p neurons are classiﬁed into a set of the lazy neurons

No

Update weight vectors
No
Yes
No

t <T
Yes
End

Fig. 3 Flowchart of LSOM learning.

Slazy at random.
(LSOM1) An input data xj is inputted to all the neurons
at the same time in parallel.
(LSOM2) We ﬁnd the winner neuron c according to Eq. (1).
If c ∈ Slazy , we perform (LSOM3). If not, we perform—2—

This neuron is the furthest to input,
namely it becomes Lazy neuron !!
Lazy neuron

Lazy neuron

Worker neurons

Worker neuron

Lazy neuron becomes
worker neuron !!

Lazy neuron
becomes winner !!

xj

xj

Fig. 4 Lazy neuron becomes worker neuron whenever it becomes winner, and anotherneuron becomes lazy.

(LSOM4).
(LSOM3) The lazy neuron, which is the winner c, is excluded from the set of the lazy neuron Slazy , and a neuron f ,which is farthest from the input data xj and is not in Slazy ,is selected to become a member of Slazy ;f = arg max{kwi − xj k},ii 6∈ Slazy .

kr i − r l k2 <= D(t),
i 6= c, i 6∈ Slazy , l ∈ Slazy },(6)

where ||r i − r l || is the neighborhood distance between map
nodes i and l on the map grid, and D(t) corresponds tothe neighborhood size, namely the quantity of the indecisive
neurons. D(t) increases with time according to the following
equation;[D(t) = Dmax

4. Experimental Results(5)

In other words, the lazy neuron becomes worker if it becomes
the winner c, and another neuron f becomes lazy shown as
Fig. 4.
(LSOM4) We ﬁnd the indecisive neurons. A set of the indecisive neurons Nlazy are the neighborhoods of each lazyneuron l in Slazy .
Nlazy = {i |where αw is the learning rate of the worker neurons, αlazy is
the learning rate of the lazy neurons, and αN is the learning
rate of the indecisive neurons, namely αlazy <= αN <= αw .
(LSOM6) The steps from (LSOM1) to (LSOM5) are repeated for all the input data.

4. 1 For Target data
We consider a 2-dimensional input data: Target data set
shown in Fig. 5(a). This data contains 770 points which has
a clustering problem of outliers [4]. Both the conventional
SOM and the proposed LSOM have 225 neurons (15 × 15)
each. p=20 neurons, namely 20% of the neurons, are classiﬁed into a set of the lazy neurons. We repeat the learning 20
times for all input data, namely T = 15400. The parameters
for the learning are chosen as follows;(For SOM)
α(0) = 0.3, σ(0) = 5.0,
(For LSOM)
αw = 0.3, αlazy = 0.03, αN = 0.15]

σ(0) = 5.0, Dmax = 5.

t
,
T

(7)

where [ ] denotes the Gauss’ notation and Dmax is a ﬁxed
parameter deciding the max value of D(t).
(LSOM5) The weight vectors of all the neurons are updatedaswi (t + 1) = wi (t) + hLc,i (t)(xj − wi (t)),

kr i − r c k2
hLc,i (t) = α exp −2σ 2 (t)

)

,

1

1

0.8

0.8

0.6

0.6

0.4

0.4

0.2

0.2

(8)

where the function hLc,i (t) is the neighborhood function of
LSOM and is described as(where we use the same learning rate: α(0) and αw , and σ(0)for the comparison and the conﬁrmation of the lazy neuron
eﬀect.

(9)0

00where α is the learning rate of LSOM and is decided by eachneuron’s character; αlazy ,
α=

αN ,

αw ,

if i = l, l ∈ Slazy
if i ∈ Nlazyotherwise,

0.2

0.4

0.6

(a)

0.8

1

0

0.2

0.4

0.6

0.8

1

(b)

Fig. 5 Learning simulation for Target data. (a) Input data.
(b) Learning result of the conventional SOM.

(10)
The simulation results of the conventional SOM is shown
—3—

1

1

1

1

0.8

0.8

0.8

0.8

0.6

0.6

0.6

0.6

0.4

0.4

0.4

0.4

0.2

0.2

0.2

0.2

0

0
0

0.2

0.4

0.6

0.8

1

0
0

0.2

(a)

0.4

0.6

0.8

1

0
0

0.2

0.4

(b)

0.6

0.8

1

0

1

1

1

0.8

0.8

0.8

0.8

0.6

0.6

0.6

0.6

0.4

0.4

0.4

0.4

0.2

0.2

0.2

0.2

0
0

0.2

0.4

(e)

0.6

0.8

1

0
0

0.2

0.4

0.6

0.8

1

0.4

0.6

0.8

1

0.6

0.8

1

(d)

1

0

0.2

(c)

0
0

0.2

0.4

(f)

0.6

0.8

1

0

0.2

(g)

0.4

(h)

Fig. 6 Learning process of proposed LSOM. Red points, blue points and white points denote worker neurons, lazy neurons and indecisive neurons, respectively. (a) Initialstate (t = 0). (b) t = 500. (c) t = 4000. (d) t = 9000. (e) t = 11000. (f) t =12000. (g) t = 13000. (h) Learning results (t = 15400).

in Fig. 5(b). We can see that the conventional SOM does not
self-organize up to all the outliers input data. Figures 6(a)(h) show the learning process of LSOM, and the ﬁnal result isshown in Fig. 6(h). We can see that LSOM can self-organize
up to all the corner data. This is because LSOM has threekinds of neurons reﬂecting the world of worker ants.
4. 1. 1 Behavior of LSOM
Let us consider the learning process and behaviors of
LSOM in more detail. In the early stage of learning as
in Fig. 6(b), many worker neurons tend to gather at concentrated input data, namely the cluster area. Meanwhile,the lazy neurons tend to move to the outside of cluster because they are not signiﬁcantly updated. However, as learning progresses, the lazy neurons self-organize other cluster
as in Fig. 6(c) because they are in diﬀerent area from theworker neurons. That is to say the lazy neurons can discover new input data since they maunder. In the middle
stage of learning as Fig. 6(d), the indecisive neurons, whoselearning rate are larger than the lazy neurons but smaller
than the worker neurons, increase. The lazy neurons andthe indecisive neurons self-organize outside the cluster. As
more learning progresses, the lazy neurons self-organize theoutliers as in Fig. 6(e). In the last stage of learning as inFigs. 6(f)-(h), the number of indecisive neurons increase further, and the map converges. From these ﬁgures, we can saythat LSOM can self-organize in every corner of input data
than the conventional SOM.
4. 1. 2 Comparison between SOM and LSOM
Furthermore, in order to compare the learning performance of LSOM with the conventional SOM numerically,

we use the following well-used two measurements.
Quantization Error Qe [1]: This measures the average distance between each input vector and its winner;
1 ∑kxj − w̄j k,
N
N

Qe =

(11)

j=1

where w̄j is the weight vector of the corresponding winner ofthe input vector xj . Therefore, the small value Qe is moredesirable.
Neuron Utilization U [5]: This measures the percentageof neurons that are the winner of one or more input vector
in the map;
1 ∑
ui ,
nm
nm

U=

(12)

i=1

where ui = 1 if the neuron i is the winner of one or more
input data. Otherwise ui = 0. Thus, U nearer 1.0 is moredesirable.
The calculated two measures are shown in Table 1. The
quantization error Qe of LSOM is smaller than the conventional SOM, and by using LSOM, the quantization error Qe
has improved by 45.3545% compared to that of SOM. In addition, the neuron utilization U of LSOM is larger than theconventional SOM, and by using LSOM, the neuron utilization U has improved by 11.6959%. This result means thatthe result of LSOM have fewer inactive neurons than theconventional SOM, and LSOM can obtain more exact map
reﬂecting the distribution state of input data.
—4—

1

1

1

0.8

0.8

0.8

0.6

0.6

0.6

0.4

0.4

0.4

0.2

0.2

0.2

0
1

0
1

0
1

1
0.5

1
0.5

0.5
0 0

1
0.5

0.5
0 0

(a)

0.5
0 0

(b)

(c)

Fig. 7 Learning simulation for Hepta data. (a) Input Data. (b) Learning result of conventional SOM. (c) Learning result of LSOM.

1

1

1

0.8

0.8

0.8

0.6

0.6

0.6

0.4

0.4

0.4

0.2

0.2

0.2

0
1

0
1

0
1

1
0.5

1
0.5

0.5
0 0

0.5

1
0.5

0 0

(a)

0.5
0 0

(b)

(c)

Fig. 8 Learning simulation for Atom data. (a) Input Data. (b) Learning simulation of
conventional SOM. (c) Learning simulation of LSOM.

Table 1 Quantization error Qe and Neuron utilization U for Target data.
Conventional SOM
LSOM
Improvement rate [%]Qe
0.01470.008045.3545U
0.76000.848911.6959

4. 2 For Hepta data
Next, we considered a 3-dimensional input data called
Hepta data set shown in Fig. 7(a), which has a clustering
problem of diﬀerent densities in clusters. The total numberof the input data N is 212, and the input data has sevenclusters.
We repeated the learning 70 times for all the input data,
namely T = 14840. The learning conditions are the same as
those used in Subsection 4. 1.
Figures 7(b) and (c) show the learning results of the conventional SOM and LSOM, respectively. We can see that
LSOM can self-organize up to edge data than the conventional SOM.
The calculated two measures are shown in Table 2. The
quantization error Qe of LSOM is smaller and the neuron
utilization U of LSOM is larger than the conventional SOM.
Because the inactive neurons have been reduced by 5.0%,

Table 2 Quantization error Qe and Neuron utilization U for
Hepta data.
Conventional SOM
LSOM
Improvement rate [%]Qe
0.02790.013850.4575

U
0.53330.56005.0

more neurons are attracted to clusters and the quantization
error Qe has been decreased by 50.4575%. It means that
LSOM can extract the feature of the input data more eﬀectively than the conventional SOM.
4. 3 For Atom data
Furthermore, we considered a 3-dimensional input data
called Atom data set shown in Fig. 8(a). This data set hasclustering problems of linear not separable, diﬀerent densities and variances. The total number of the input data N is
800, and the input data has two clusters.
We repeated the learning 20 times for all input data,
namely T = 16000. The learning conditions are the sameas those used in Subsection 4. 1.
The learning results of the conventional SOM and LSOM
are shown in Figs. 8(b) and (c). We can see that LSOM canself-organize edge data more eﬀective than the conventional
SOM.
—5—

Table 3 Quantization error Qe and Neuron utilization U for
Atom data.
Qe
U
Conventional SOM
0.04930.7467LSOM
0.03140.8267
Improvement rate [%] 36.2946 10.7143

Table. 3 shows the calculated two measures. The quantization error Qe of LSOM is smaller than the conventional SOM,
and the quantization error Qe has improved 36.2946% fromusing the conventional SOM. In addition, the neuron utilization U of LSOM is larger than the conventional SOM, andby using LSOM, the neuron utilization U has improved by
10.7143%. From these results, we can say that LSOM, which
includes the lazy neurons, can self-organize more eﬀectively
than the conventional SOM which contains only worker neurons.

5. Conclusions
In this study, we have proposed the Lazy Self-Organizing
Map (LSOM), which reﬂects the world of worker ants. LSOM
contains three kinds of neurons: worker neurons, lazy neurons and indecisive neurons which are neighborhoods of thelazy neurons. We have investigated the behaviors of LSOM
by applying it to some input data set. We have conﬁrmedthat LSOM can self-organize in every corner of the input
data and can obtain more exact map reﬂecting the distribution state of input data than the conventional SOM. Fromthese results, we can say that LSOM, which includes the lazyneurons, can self-organize more eﬀectively than the conventional SOM which contains only worker neurons.
References[1] T. Kohonen, Self-Organizing Maps, Berlin, Springer,
vol. 30, 1995.
[2] J. Vesanto and E. Alhoniemi, “Clustering of the SelfOrganizing Map,” IEEE Trans. Neural Networks, vol. 11,
no. 3, pp. 586–600, 2002.
[3] H. Hasegawa, “Optimization of GROUP Behavior,” Japan
Ethological Society Newsletter, no. 43, pp. 22–23, 2004.
[4] A. Ultsch, “Clustering with SOM: U*C”, Proc. Workshop
on Self-Organizing Maps, pp.75-82, 2005.
[5] Y. Cheung and L. Law, “Rival-Model Penalized SelfOrganizing Map,” IEEE Trans. Neural Networks, vol. 18,
no. 1, pp. 289–295, 2007.

—6—