IEICE TRANS. FUNDAMENTALS, VOL.E90–A, NO.4 APRIL 2007

865

PAPER

Competing Behavior of Two Kinds of Self-Organizing Maps
and Its Application to Clustering
Haruna MATSUSHITA†a) , Student Member and Yoshifumi NISHIO†b) , Member

SUMMARY
The Self-Organizing Map (SOM) is an unsupervised neural network introduced in the 80’s by Teuvo Kohonen. In this paper, we
propose a method of simultaneously using two kinds of SOM whose features are diﬀerent (the nSOM method). Namely, one is distributed in thearea at which input data are concentrated, and the other self-organizes thewhole of the input space. The competing behavior of the two kinds of
SOM for nonuniform input data is investigated. Furthermore, we show its
application to clustering and confirm its eﬃciency by comparing with the
k-means method.
key words: self-organizing maps, clustering, data mining, data extraction

1.

Introduction

In data mining, clustering is essential and important. The
k-means method is known as the representative method because of its simplicity [1]. However, when the data contains
much noise, it is diﬃcult to extract only the cluster exactly
using the k-means method. Then, the Self-Organizing Map(SOM) has attracted attention in recent years. SOM is an unsupervised neural network introduced by Kohonen in 1982
[2] and is a model simplifying self-organization process ofthe brain. SOM obtains a statistical feature of input data andis applied to a wide field of data classifications. Althoughmany methods to extract clusters by using SOM have been
proposed [3]–[6], it seems to be very diﬃcult to constructa simple method using SOM for universal input data. Furthermore, since we can accumulate a large amount of data
including useless information nowadays, it is important toinvestigate various extraction methods of clusters from data
including much noise.
In this study, we propose a method simultaneously using two kinds of SOM whose features are diﬀerent (thenSOM method). Namely, one self-organizes the area atwhich input data are concentrated, and the other selforganizes the whole of the input space.
In Sect. 2, we explain the diﬀerence of the two kinds
of SOM with the learning algorithm, and the competing behavior of the two kinds of SOM is investigated. In Sect. 3,
we apply the two kinds of SOM to clustering. For 2 and 3dimensional input data including much noise (correspondManuscript received January 16, 2006.
Manuscript revised August 22, 2006.
Final manuscript received January 4, 2007.
†
The authors are with the Department of Electrical and Electronic Engineering, Tokushima University, Tokushima-shi, 770–
8506 Japan.
a) E-mail: haruna@ee.tokushima-u.ac.jpb) E-mail: nishio@ee.tokushima-u.ac.jp
DOI: 10.1093/ietfec/e90–a.4.865

ing to useless information), clustering ability is evaluatedboth visually and quantitatively using a correct answer rate.
Furthermore, the results are compared with those obtainedby the k-means method. We also apply the two kinds of
SOM to 5-dimensional input data including much noise andconfirm the clustering ability for higher-dimensional data.
Furthermore, we carry out simulation for real data sets (theIris plant data [7]), and we evaluate the eﬀectiveness of clustering the real data using the two kinds of SOM.
2.

Two Kinds of SOM

In this study, we propose a method simultaneously using
two kinds of SOM (the nSOM method), namely, one selforganizes the area at which input data are concentrated, andthe other self-organizes the whole of the input space. We
call the former SOML and the latter SOMG .
2.1 Learning Algorithm
We explain the learning algorithm of the two kinds of SOM.
In order to apply the two kinds of SOM to clustering applications, we use totally n SOMs, that is one SOMG and(n − 1) SOML ; namely SOML1 , SOML2 , · · · , SOML(n−1) .
In each SOM, m neurons are arranged as a regular 2dimensional grid shown in Fig. 1. The range of the elements of d-dimensional input data x j = (x j1 , x j2 , · · · , x jd )( j = 1, 2, · · · , N) are assumed to be from 0 to 1. Each neuronof each SOM has a d-dimensional weight vector, namely,
the same dimension as input data. We call the values of theweight vectors of all neurons in a SOM as the state of the
SOM. nSOM learns according to the following steps from(nSOM1) to (nSOM11).
[PHASE 1](nSOM1) The initial values of all the weight vectors wLl of
SOMLl (l = 1, 2, · · · , n − 1) are given between 0 and 1 at
random. The initial values of all the weight vectors wG of
SOMG are given around the center of the input space at random, for example uniform between 0.45 and 0.55.
[PHASE 2](nSOM2) An input data x j is inputted to all the neurons of
SOMG and SOMLl simultaneously in parallel.
(nSOM3) The distance between x j and the weight vector
wGi = (wGi1 , wGi2 , · · · , wGid ) (i = 1, 2, · · · , m) of the neuron
i of SOMG , and the distance between x j and the weight vector wLl i = (wLl i1 , wLl i2 , · · · , wLl id ) of the neuron i of SOMLlc 2007 The Institute of Electronics, Information and Communication Engineers
Copyright IEICE TRANS. FUNDAMENTALS, VOL.E90–A, NO.4 APRIL 2007

866

Fig. 1

Structure of nSOM.

are calculated. The internal activity degrees netG ij and netLl ij
are obtained as;netG ij = wG i − x j −1 ,netLl ij = wLl i − x j −1 ,(1)where  ·  is the distance measure, in this study, Euclidian
distance.
(nSOM4) The winner c for x j is the neuron with the maximum internal activity degree in all netGj and netL lj .
(nSOM5) If the winner c is a neuron in SOMG , the weightvectors of all the neurons of SOMG are updated as;
wGi (t + 1) = wGi (t) + hGc,i (t)(x j − wGi (t)),(2)

where hGc,i (t) is the neighborhood function of SOMG .
While, if the winner c is in SOMLl , the weight vectorsof the neurons of SOMLl are updated as;
wLl i (t + 1) = wLl i (t) + hLc,i (t)(x j − wLl i (t)),(3)

where hLc,i (t) is the neighborhood function of SOML . The
neighborhood functions hGc,i (t) and hLc,i (t) are importantfunctions deciding the behaviors of nSOM and are explainedin the next subsection.
(nSOM6) The steps from (nSOM2) to (nSOM5) are repeated for all the input data, namely from j = 1 to j = N.
[PHASE 3](nSOM7) Furthermore, only SOMG learns with the time reset as t = 0.
(nSOM8) An input data is inputted to all the neurons similarly to the step (nSOM2).
(nSOM9) The internal activity degree netLl ij is calculated
similarly to the step (nSOM3).
(nSOM10) If the maximum value of netLl ij is smaller than agiven threshold value 1/ε (this means that the distance between the input data and SOMLl is larger than ε), the weightvectors of the neurons of SOMG are updated as;
wGi (t + 1) = wGi (t) + hGsc,i (t)(x j − wGi (t)).

(4)(nSOM11) The steps from (nSOM8) to (nSOM10) are repeated for all the input data.
2.2 Neighborhood Function
In the learning algorithm of nSOM, the diﬀerence between

the neighborhood functions of SOMG and SOML plays a keyrole to decide their behaviors.
The neighborhood functions for SOMG and SOML aredescribed as follows;⎛⎞⎜⎜⎜ ri − rc 2 ⎟⎟⎟⎜⎟⎠ ,hG c,i (t) = αG (t) exp ⎝−2σG2 (t)⎛⎞⎜⎜⎜ ri − rc 2 ⎟⎟⎟⎟⎠ ,(5)hLc,i (t) = αL (t) exp ⎜⎝−2σ2L (t)where αG (t) and αL (t) are the learning rate, ri is the coordinates of neuron i on the map grid, ri − rc  is the distance between map nodes i and c on the map grid, and σG (t)and σL (t) correspond to the width of the neighborhood functions.
In order to give diﬀerent features to SOMG and SOML ,we set the following schedule functions for αG (t), σG (t),αL (t), and σL (t).
αG (t) = αG (0) (1 − t/T ) ,σG (t) = σG (0) 1 − (t/T )2 ,1αL (t) = αL (0) 1 − (t/T ) 2 ,σL (t) = σL (0) (1 − t/T ) ,(6)

where T is the learning length.
Figure 2 shows an example of the neighborhood functions for αG (0) = 0.9, αL (0) = 0.5, σG (0) = σL (0) = 4,
and T = 6400. The lines G1 and L1 show the case that thevalue of ri −√rc  is zero, while G2 and L2 show the case ofri − rc  = 4 2.
The neighborhood function for SOMG in the [PHASE
3] is given as follows;⎛⎞⎜⎜ ri − rc 2 ⎟⎟⎟⎟.
hGsc,i (t) = αGs (t) exp ⎜⎜⎝−
(7)
2 (t) ⎠
2σGs
αGs (t) and σGs (t) decrease with time according to the following equations;αGs (t) = αGs (0) (1 − t/T s ) ,σGs (t) = σGs (0) (1 − t/T s ) ,where T s is the learning length since the step (nSOM7).

(8)

MATSUSHITA and NISHIO: COMPETING BEHAVIOR OF TWO KINDS OF SELF-ORGANIZING MAPS

867

Fig. 4

Fig. 2

An example of neighborhood functions.

(a)Fig. 3(b)(c)(a) Input data. (b) Initial state of SOMG . (c) Initial state of SOML .

2.3 Simulation Results
As explained in the previous subsections, the main diﬀerences between SOMG and SOML are only the initial states
and the neighborhood functions. However, these diﬀerences
cause the interesting competing behavior of nSOM.
Figure 3 shows an example of input data and initialstates of SOMG and SOML for the case of n = 2 (namely
the number of SOML is only one). Input data in Fig. 3(a)include 2-dimensional 1600 points (N = 1600). Each SOM
has 100 neurons (10 × 10).
The parameters of the learning are chosen as follows;αG (0) = 0.9, αL (0) = 0.5, σG (0) = 2, σL (0) = 4,
αGs (0) = 0.5, σGs (0) = 2, ε = 0.02.
After repeating [PHASE 2] four times, furthermore(a)(b)
Simulation results. (a) After [PHASE 2]. (b) After [PHASE 3].

[PHASE 3] is repeated four more times.
The simulation result is shown in Fig. 4. Figures 4(a)and (b) show the states after [PHASE 2] and [PHASE 3],
respectively. The network at the upper left corner is SOML
and the other network is SOMG .
Because only one neuron of all the neurons in theboth SOMs can be a winner for one input data according
to (nSOM4), the two networks compete with each otherin [PHASE 2] as shown in Fig. 4(a). In the early stage of[PHASE 2], only SOML actively moves according to the input data, because the initial state of SOML covers the whole
input space. Hence, SOML tends to move to the area where
the input data are dense. In the late stage of [PHASE 2],
SOML will not make a large move any more, because thelearning rate αL (t) decreases rapidly according to Eq. (6).
While SOMG actively moves all over the whole space except the area occupied by SOML , because SOML stays in
the limited area and the width of the neighborhood function
σG (t) decreases slowly according to Eq. (6).
In the [PHASE 3], SOMG covers the whole input spacebeyond the area occupied by SOML as shown in Fig. 4(b).
The reasonable value of ε is from 5 to 50% of the size ofcluster.
3.

Application to Clustering

The concept using nSOM can be exploited to extract the data
only in clusters of the input data including much noise, because SOML can find such areas by themselves.
3.1 2-Dimensional Input DataFirst, we consider 2-dimensional input data as shown in
Fig. 5(a). The input data is generated artificially as follows. The total number of the input data is 1600. 25% of
the input data are distributed within a range from 0.2 to 0.3
horizontally and from 0.7 to 0.8 vertically, and these data
are called the cluster C1 . 50% of the input data are distributed in another cluster C2 , whose horizontal-values follow the normal distribution N(0.7, 0.04), and the verticalvalues N(0.2, 0.0016). The remaining 25% of the input data
are distributed between 0 and 1 at random.
We use one SOMG and two SOML . Each SOM has 100
neurons (10×10), namely 3 SOMs have totally 300 neurons.

IEICE TRANS. FUNDAMENTALS, VOL.E90–A, NO.4 APRIL 2007

868

(a)

(a)

(b)
(c)
Fig. 5 Clustering of 2-dimensional input data. (a) Input data. (b) Simulated result after [PHASE 2]. (c) Simulated result after [PHASE 3].

(b)(c)
Fig. 7 Extraction of clusters by k-means method. (a) Cluster 1. (b) Cluster 2. (c) Noises.
Table 1

Correct answer rate [%] for 2-dimensional input data.

nSOM methodk-means method

Fig. 62.

(a)(b)
Extraction of clusters by nSOM method. (a) Cluster 1. (b) Cluster

The parameters of the learning are chosen as follows;αG (0) = 0.9, αL (0) = 0.5, σG (0) = σL (0) = 4,
αGs (0) = 0.7, σGs (0) = 2, ε = 0.05.
After repeating [PHASE 2] four times, furthermore[PHASE 3] are repeated four more times.
The simulation results after [PHASE 2] and [PHASE
3] are shown in Figs. 5(b) and (c), respectively. We can seethat two SOML stay around the two clusters. If we do not
use SOMG (namely, we used only two SOML ), each SOML
covers not only the cluster but also much noise.
In order to extract the input data only in the clusters,
we calculate the distance between the input data x j and wLl
in SOMLl after [PHASE 2]. (Actually, for the purpose of
the clustering, we do not need [PHASE 3].) If the calculated distance is smaller than a threshold R, the input data x j
is classified into the cluster corresponding to SOMLl . Figures 6(a) and (b) show the input data classified into the clusters corresponding to SOML1 and SOML2 , respectively, for

C1
86.80
79.95

C2
91.28
70.18

R = 0.05. As we can see from the figures, SOML can successfully extract the clusters.
The clustering for the input data including much noise
becomes increasingly important. However, the method of
extracting only clusters from the input data including muchnoise, has not been defined well. Although, the k-means
method is known to be not useful for the data includingmuch noise, we carry out the k-means method for the same
input data for the comparison. Figure 7 shows the results
obtained by using the k-means method, where the numberof the cluster is set as k = 3 (which corresponds to 2 clusters and noise). We can see that the clusters obtained by the
k-means method include much noise.
In order to evaluate the clustering ability of nSOM
quantitatively, we define the correct answer rate RC I as follows;
RC I =

Nr − Ne
,
NC I

(9)

where NC I is the true number of the input data within thecluster C I , Nr is the obtained number of the desired input
data within C I , and Ne is the obtained number of undesired
input data outside C I . The calculated results are summarizedin Table 1. We can evaluate the eﬀectiveness of the method
using nSOM by this index value.

MATSUSHITA and NISHIO: COMPETING BEHAVIOR OF TWO KINDS OF SELF-ORGANIZING MAPS

869

(a)

(b)

(c)
(d)
Fig. 8 Clustering results by nSOM with various sizes. (a) m = 3. (b) m =8. (c) m = 15. (d) m = 35.

3.2 SOM Size Eﬀect on the Clustering
In this section, we explain the SOM size eﬀect on the clustering results.
Figure 8 shows the clustering results by nSOM withvarious sizes for 2-dimensional input data. We can obtain effective results of clustering when we use nSOM with appropriate size which is not too small or too large, as Figs. 8(b)and (c). However, if the SOM size is too small for the number of input data, as Fig. 8(a), we do not get the eﬀect ofneighborhood functions which is very important feature ofnSOM. Conversely, for too large as Fig. 8(d), SOML selforganizes unnecessary noises as well as clusters.

(a)

(b)
(c)
Fig. 9 Clustering of 3-dimensional input data. (a) Input data. (b) Extracted cluster by SOML1 . (c) Extracted cluster by SOML2 .

Table 2

Correct answer rate [%] for 3-dimensional input data.

nSOM methodk-means method

C1
89.00
61.91

C2
86.15
60.19

by random Gaussian data as shown in Table 3. The parameters of the learning are chosen as;
αG (0) = 0.9, αL (0) = 0.6, σG (0) = σL (0) = 4,
αGs (0) = 0.7, σGs (0) = 2, ε = 0.2, R = 0.2.
The correct answer rates are summarized in Table 4.
We can say that the method of using nSOM are eﬀective forhigher-dimensional input data.

3.3 3-Dimensional Input Data
3.5 Application to Real Data Sets
Next, we carry out simulation for 3-dimensional input data
shown in Fig. 9(a). The input data include two clusters andmuch noise from the clusters.
Figures 9(b) and (c) show the extracted clusters by thenSOM method. We can confirm that the noises are removed
by SOMG and only the cluster part can be extracted verywell.
The correct answer rates are summarized in Table 2.
We can confirm the clustering ability using nSOM.
3.4 5-Dimensional Input Data
Furthermore, we performed the simulation for 5-dimensional input data of 1600 points. These data include fourclusters and much noise, and the four clusters are generated

We apply the proposed method to real data sets. We use theIris plant data [7] as real data. This data is one of the best
known databases to be found in pattern recognition literatures [8]. The data set contains three clusters of 50 instances
respectively, where each class refers to a type of iris plant.
The number of attributes is four as the sepal length, the sepal
width, the petal length and the petal width, namely, the input
data are 4-dimension. The three classes correspond to Iris
setosa, Iris versicolor and Iris virginica, respectively. Irissatosa is linearly separable from the other two, however Iris
versicolor and Iris virginica are not linearly separable fromeach other. Furthermore, we add 50 noises in the Iris data.
In fact the input data is 4-dimensional normalized data of
200 points (50 in each of three classes and noises) shown inIEICE TRANS. FUNDAMENTALS, VOL.E90–A, NO.4 APRIL 2007

870
Table 3
No.
C1
C2
C3
C4
noise

Table 4

Mean value
Variance
Mean value
Variance
Mean value
Variance
Mean value
Variance
random1
0.8
0.00160.35
0.00360.6
0.00040.2
0.0016
0.0–1.05-dimensional Gaussian data.
Dimension
2
3
4
0.3
0.7
0.2
0.00810.00360.00090.8
0.3
0.7
0.00040.01
0.00810.9
0.1
0.1
0.00160.00090.0025
0.1
0.8
0.4
0.01
0.00040.0225
0.0–1.0
0.0–1.0
0.0–1.0

The correct answer rate [%] of 5-dimensional input data.

nSOM methodk-means method

C1
89.84
54.69

C2
94.22
73.86

C3
98.39
85.48

5
0.7
0.0081
0.2
0.00360.9
0.00090.3
0.0025
0.0–1.0

Probability [%]
15
20
15
20
30

(1)

C4
96.33
73.33

(2)

(a)
(b)
Fig. 10 Iris data. Circle (◦), plus (+), square () and dots (.) correspond to Iris setosa, Iris versicolor, Iris virginica and noises, respectively.
(a) Horizontal axis is the sepal length and vertical axis is the petal length.
(b) Horizontal axis is the petal width and vertical axis is the sepal width.
(3)

Fig. 10.
We use one SOMG and three SOML . Each SOM has
16 neurons (4 × 4), namely 4 SOMs have totally 64 neurons.
The parameters of the learning are chosen as follows;αG (0) = αL (0) = 0.9, σG (0) = σL (0) = 2, R = 0.1.
We repeat [PHASE 2] four times.
Figure 11 shows simulation results, and the correct answer rates are summarized in Table 5. We can say thatnSOM is eﬀective method in the clustering of the real data.
4.

Conclusions

In this study, we have proposed the method simultaneously
using two kinds of SOM whose features are diﬀerent (thenSOM method). We have investigated its competing behavior caused by the diﬀerence of the initial states and theneighborhood functions. Furthermore, we have applied thetwo kinds of SOM to clustering of data including muchnoise and have confirmed its eﬃciency.
References[1] Y. Linde, A. Buzo, and R. Gray, “An algorithm for vector quantizer
design,” IEEE Trans. Commun., vol.28, no.1, pp.84–85, 1980.
[2] T. Kohonen, Self-Organizing Maps, Springer, Berlin, 1995.

(a)(b)
Fig. 11 Simulation results of Iris data. (a) Horizontal axis is the sepal
length and vertical axis is the petal length. (b) Horizontal axis is the petal
width and vertical axis is the sepal width. (1) Iris setosa. (2) Iris versicolor.
(3) Iris virginica.

Table 5nSOM methodk-means method

The correct answer rate [%] of Iris data.
Iris setosa94
92

Iris versicolor
90
44

Iris virginica
82
60[3] Y. Cheng, “Clustering with competing self-organizing maps,” Proc.
IJCNN’92, vol.IV, pp.785–790, 1992.
[4] W. Wan and D. Fraser, “M2dSOMAP: Clustering and classificationof remotely sensed imagery by combining multiple Kohonen selforganizing maps and associative memory,” Proc. IJCNN’93, vol.III,

MATSUSHITA and NISHIO: COMPETING BEHAVIOR OF TWO KINDS OF SELF-ORGANIZING MAPS871

pp.2464–2467, 1993.
[5] J. Vesanto and E. Alhoniemi, “Clustering of the self-organizing map,”
IEEE Trans. Neural Netw., vol.11, no.3, pp.586–600, 2000.
[6] I. Lapidot, H. Guterman, and A. Cohen, “Unsupervised speaker recognition based on competition between self-organizing maps,” IEEE
Trans. Neural Netw., vol.13, no.4, pp.877–887, 2002.
[7] UCI Repository of Machine Learning Databases and Domain
Theories, FTP address: ftp://ftp.ics.uci.edu/pub/machine-learningdatabases[8] R.A. Fisher, “The use of multiple measurements in taxonomic problems,” Annual Eugenics, no.7, part II, pp.179–188, 1936.

Haruna Matsushita
was born in Tokushima, Japan, in 1982. She received the B.E.
degree from Tokushima University, Tokushima,
Japan, 2005. She is currently working towards
M.E. degree at the same university. Her researchinterests include theory and application of SelfOrganizing Maps. She is a student member of
the IEEE.

Yoshifumi Nishio
received B.E., M.E.,
and Ph.D. degrees in electrical engineering from
Keio University, Yokohama Japan, in 1988,
1990, and 1993, respectively. In 1993, he joined
the Department of Electrical and Electronic Engineering at Tokushima University, Tokushima
Japan, where he is currently an Associate Professor. From May 2000 he spent a year in the
Laboratory of Nonlinear Systems (LANOS) at
the Swiss Federal Institute of Technology Lausanne (EPFL) as a Visiting Professor. His research interests include analysis and application of chaos in electrical circuits, analysis of synchronization in coupled oscillatory circuits, development of analyzing methods for nonlinear circuits, theory and applicationof cellular neural networks, and neural network architecture. He was the
Chair of the IEEE CAS Technical Committee on Nonlinear Circuits andSystems (NCAS) during 2004–2005 and is currently the Steering Committee Secretary of the IEICE Research Society of Nonlinear Theory and its
Applications (NOLTA). He is serving as an Associate Editor for the IEEE
Transactions on Circuits and Systems Part I and the RISP Journal of Signal
Processing. He is a senior member of the IEEE, and a member of the RISP.