2009 IEEE Workshop on
Nonlinear Circuit Networks, Tokushima, Dec 11-12, 2009

Behavior of Community Self-Organizing Mapfor Clustering and Data Extraction
Taku Haraguchi, Haruna Matsushita and Yoshifumi Nishio
Department of Electrical and Electronic Engineering,
Tokushima University
Email: {taku, haruna, nishio}@ee.tokushima-u.ac.jp

Abstract— In the previous study, we have proposed the Community Self-Organizing Map (CSOM) that the neurons create
some neuron-community according to their winning frequencies.
In this study, we apply CSOM to clustering and data extraction
for various input data including a lot of noises, and we investigate
its numerical efficiency by using correct answer rate. We confirm
that CSOM creates some communities and obtain effective resultsfor data extraction.

I. I NTRODUCTION
In data mining, clustering is one of typical analysis techniques and is studied for many applications, such as astatement, a pattern recognition, an image analysis and soon. Then, the Self-Organizing Map (SOM) [1] has attracted
attention for the study on clustering in recent years. SOM is anunsupervised neural network introduced by Kohonen in 1982
and is a simplified model of the self-organization process ofthe brain. SOM obtains statistical feature of input data and isapplied to a wide field of data classifications [2]−[8]. SOM
can classify input data according to similarities and patterns
which are obtained by the distance between neurons and somevisualization methods based on SOM were proposed [9]-[13].
On the other hand, in the real world, the amount and thecomplexity of data increase from year to year. Therefore, it isimportant to investigate various extraction method of clusters
from data including a lot of noises.
Meanwhile, the real world is competitive society andhuman-beings belong to sub-society called “Community”. It isbased on definition that the human-beings are social animals
introduced by Aristotle. The social animal creates a society
and lives in the society. Furthermore, the social animals inthe society are centered around a leader and create the community. Additionally, there are also human-beings excluded
from the community. The phenomenon actually take place inthe real world. On the other hand, the neuron’s world is also
competitive society like the real world. Therefore, we consider
that the neurons are centered around a leader and create thecommunity. Namely, the neurons are also the social animal.
In previous study, we have proposed the Community SelfOrganizing Map (CSOM) [14] that the neurons create someneuron-community according to their winning frequency, andthe neurons, which is not satisfied the condition, are removed
from the community including these after all leaning.
In this study, we apply CSOM to clustering and data
extraction for various input data including a lot of noises.

Furthermore, we investigate its numerical efficiency by using
correct answer rate.
II. C OMMUNITY S ELF -O RGANIZING M AP
In the previous study, we have proposed a Community
SOM (CSOM) algorithm. The important feature of CSOM isthat the neurons create some neuron-community according totheir winning frequency. In other words, in CSOM algorithm,
the winner, which satisfies the condition for the winningfrequency, and its neighborhood neurons, which satisfy thesame condition, create kth community Ck . In the community
Ck , a leader lk is a neuron that has become the winner mostfrequently among the all neurons belonging to Ck . Because
in the human society, the human-beings also creates somecommunity. This phenomenon has tendency that human-beings
gather around a leader.
A. Learning Algorithm
We explain the learning algorithm of CSOM in detail.
CSOM has a two-layer structure of the input layer and thecompetitive layer as the conventional SOM. In the input layer,
there are d-dimensional input vectors xj = (xj1 , xj2 , · · · , xjd )(j = 1, 2, · · · , N ). In the competitive layer, M neurons arearranged as a regular 2-dimensional grid. Each neuron has aweight vectors wi = (wi1 , wi2 , · · · , wid ) (i = 1, 2, · · · , M )with the same dimension as the input vector. A winningfrequency Wi is associated with each neuron and is set to zero
initially: Wi = 0. The number of members in each community
Ck and the number of community n are zero. Before learning,
the all neurons do not belong to any community, however, they
gradually belong to some community with learning.
(CSOM1) Input an input vector xj to all the neurons simultaneously in parallel.
(CSOM2) Find a winner c by calculating a distance between
the input vector xj and the weight vector wi of each neuron i;c = arg min{kwi − xj k},i(1)where k · k is the distance measure, in this study, we use
Euclidean distance.
(CSOM3) Updated the weight vectors of all the neurons as- 98 -wi (t + 1) = wi (t) + hc,i (t)(xj − wi (t)),(2)

where t is the learning step. hc,i (t) is called the neighborhood
function and is described as()kr i − r c k2hc,i (t) = α(t) exp −,(3)
2σ 2 (t)where r i and r c are the vectorial locations on the display
grid, α(t) is called the learning rate, and σ(t) corresponds tothe width of the neighborhood function. Both α(t) and σ(t)decrease monotonically with time, in this study, we use()()tt
α(t) = α(0) 1 −, σ(t) = σ(0) 1 −,TT(5)Tmin t(6)) .
T M(CSOM5) Find the community Ck including the winner c.
If winner c does not belong to any community, create a newcommunity, nnew = nold + 1, and affiliate the winner c to newcommunity Ck as c ∈ Ck (where k = nnew ). If not, c remains
in its community Ck .
(CSOM6) Find a leader lk which has become the winner mostfrequently among the all neurons belonging to Ck , according
to Eq. (7) as Fig. 1.
Wth (t) = (1 −ii ∈ Ck .

(7)

Community Ck

2015

1-neighborhoodsEx.) Wth (t) = 9

This neuron does not satisfy thecondition of winning frequency.

Fig. 2. How to update community Ck . Number in each neuron denotes
its winning frequency Wi . The winner’s 1-neighborhood neurons with higher
winning frequency than Wth (t) belong to community Ck . The neuron with
Wi = 5, which is lower winning frequency than Wth (t), belongs to nocommunity.

(CSOM8) Compare the winning frequencies of two leaders between the community including the winner and thecommunity including winner’s neighborhood neuron. Lossof generality, assume that the winner c belongs to C1 andits neighborhood neuron belongs to C2 . The leaders of C1
and C2 are assumed as l1 and l2 , respectively. If Wl2 ≥
Wl1 , the neighborhood neuron keeps on belonging to C2 asFig. 3(a). If not, the neighborhood neuron belonging to C2
are absorbed into C1 as Fig. 3(b). Then, in a specific case,if the neighborhood neuron is the leader l2 in the community
C2 , all the neurons belonging to C2 are absorbed into C1 anddecrease the number of communities as nnew = nold − 1.
(CSOM9) Repeat the steps from (CSOM1) to (CSOM8) for
all the input data.
(CSOM10) After all learning are finished, check whether
Wi > 0.8 × Wth (T ) for each particle i. If it is not satisfied,remove the particle i from the community including it.
III. E XPERIMENT RESULTS
A. Application to Clustering

10
20c

5

Leader lk

12Winner neuron10(4)Evaluate whether the winner c satisfies the conditions of thewinning frequency to update the community informations. If
Wc > Wth (t) is satisfied, perform (CSOM5). If not, perform(CSOM9) without updating the community. Wth (t) is thethreshold value and increases with learning as

lk = arg max{Wi },Winner neuron cand its community Ck

12where T is the maximum number of the learning.
If t ≥ Tmin is satisfied, perform (CSOM4). If not, perform(CSOM9). Tmin is fixed parameter and the minimum numberof the learning in creating community.
(CSOM4) Increase the winning frequency of the winner c by
Wcnew = Wcold + 1.

than Wth (t), in 1-neighborhoods of the winner c, then consider
whether they belong to any community. If this neighborhood
neuron belongs to any community, perform (CSOM8). If
not, affiliate it to the community Ck including the winnerc in Fig. 2, update the leader lk as (CSOM6), and perform(CSOM9).

155Community CkFig. 1. How to update leader lk in community Ck . Number in each neuron
denotes its winning frequency Wi . The neuron with Wi = 20, which is thehighest winning frequency among the neurons in the community Ck , becomesthe leader lk .

(CSOM7) Find neurons, whose winning frequency are higher

We consider 2-dimensional input data containing threeclusters and a lot of noises as shown in Fig. 4(a). The total
number of the input data N is 1000, and 200 data are randomly
distributed within a range from 0 to 1. The variance of thecluster is different, respectively. The conventional SOM andCSOM have 100 neurons (10×10), respectively. We repeat thelearning 15 times for all the input data, namely T = 15000.
The parameters for the learning of the conventional SOM andCSOM are chosen as follows;T.
2
The simulation results of the conventional SOM and CSOM
are shown in Figs. 4(b) and (c), respectively. In Fig. 4(c),- 99 -α(0) = 0.3, σ(0) = 3.5, Tmin =Leader l1
Winner neuron c

Leader l2

C1Leader l1

C220112512151620101118121516

Wl1 < Wl2

Community C1Winner’s
neighborhood neuron

C2

Community C2Community C2

10C1

Leader l2

Winner neuron c

Wl1 > Wl2

Community C1

C2
Neighborhood neuron
keeps on belonging to C2 .

Neighborhood neuron
absorb into C1 .

Winner’s
neighborhood neuron

(a)

(b)

Fig. 3. How to determine whether a community C1 absorb a community C2 including a neighborhood neuron. Number in each neuron denotes its winningfrequency Wi . The leader l1 and l2 are a neuron with the highest winning frequency in the community C1 and in the community C2 , respectively. (a) As thewinning frequency Wl1 = 20 of the leader l1 is lower than the winning frequency Wl2 = 25 of the leader l2 , the neighborhood neuron keeps on belonging
to C2 . (b) As Wl1 = 20 is higher than Wl2 = 18, the neighborhood neuron is absorbed into C1 .

we can see that the number of communities is three (N, Fand ¥ mean the each community), and it is the same as thenumber of clusters. It means that only the neurons, which
self-organize the area where the input data are concentrated,create the communities. Furthermore, the neurons belonging
to the largest community, which is the largest in the number ofneurons belonging to the community, self-organize the largest
cluster. Therefore, we can see the number of clusters and therough condition by investigating the number of communities
and the size.

RCI as follows [8];RCI =(i = 1, 2, · · ·),(8)

where NCI is the true number of the input data within thecluster CI , NrI is the obtained number of the desired input
data within CI , and NeI is the obtained number of undesired
input data out of CI .
TABLE IC ORRECT ANSWER RATE [%] FOR 2- DIMENSIONAL INPUT DATA .

B. Application to Data Extraction
Next, we carry out the extraction of cluster from the resultsof the conventional SOM and CSOM as Figs. 4(b) and (c).
The extraction method is relatively simple as follows. In theconventional SOM, after learning, the input data, which iswithin a radius of R from all neurons on the map, are classified
into the cluster. In CSOM, after learning, the input data, which
is within a radius of R from all neurons belonging to each
community on the map, are classified into the cluster.
The extraction result of the conventional SOM is shown inFig. 5(a), and the extraction results by using the respective
communities and the only largest community in CSOM areshown in Figs. 5(b) and (c), respectively (R = 0.05). In
Fig. 5(a), we can see that the cluster obtained by the conventional SOM includes a lot of noises. In other words, theconventional SOM obtains the unnecessary data. In CSOM,
as the neurons belonging to any community self-organize anycluster, the results as Fig. 5(b) obtain three clusters and hardly
include the noises. Besides, as the neurons self-organizing anycluster create one community at the area, we can obtain thelargest cluster by extracting the largest community as Fig. 5(c).

NrI − NeI,
NCI

MethodConventional SOM
CSOM

NeI
96
43

NrI
789
755

Correct answer rate [%]
86.6
89.0

Table I shows the correct answer rate RCI of the conventional SOM and CSOM for the 2-dimensional data, respectively. From this table, we can see that the correct answer rate
RCI and the obtained number of undesired input data NeI ofCSOM is better value than them of the conventional SOM. It
means that CSOM hardly include the noises and can exactly
extract the clusters than the conventional SOM. Therefore, we
can that CSOM obtains numerically effective result than theconventional SOM.
IV. C ONCLUSIONSIn this study, we have applied CSOM to the clustering andthe data extraction for various input data including a lot ofnoises. We have confirmed that the number of communities inCSOM is the same as the number of clusters, and CSOM canobtain visually and numerically effective results for the data
extraction.
R EFERENCES

C. Numerical Analysis for Data Extraction
In order to investigate the ability of the conventional SOM
and CSOM quantitatively, we define the correct answer rate

[1] T. Kohonen, Self-Organizing Maps, Berlin, Springer, vol. 30, 1995.
[2] Y. Cheng, “Clustering with Competing Self-Organizing Maps,” Proc. ofIJCNN’92, vol. IV, pp. 785-790, 1992.

- 100 -11

1

0.8

0.8

0.8

0.6

0.6

0.6

0.4

0.4

0.4

0.2

0.2

0.2

00

0.2

0.4

0.6

0.8

1000.2

0.4

(a)

0.6

0.8

1000.2

0.4

(b)

0.6

0.8

1(c)

Fig. 4. Learning simulation for 2-dimensional data by using the conventional SOM and CSOM. N, F and ¨ denote the largest community C1 , the second
largest community C2 and the third largest community C3 , respectively. (a) Input data. (b) Learning result of the conventional SOM. (c) Learning result ofCSOM.

1110.8

0.8

0.8

0.6

0.6

0.6

0.4

0.4

0.4

0.2

0.2

0.2

00

0.2

0.4

0.6

0.8

1000.2

0.4

(a)

0.6

(b)

0.8

100

0.2

0.4

0.6

0.8

1(c)

Fig. 5. Extraction results of clusters. (a) Clusters extracted by using all the neurons in the conventional SOM. (b) Clusters extracted by using all the neurons
belonging to the respective community in CSOM. (c) Clusters extracted by using the largest community C1 in CSOM.

[3] W. Wan and D. Fraser, “M2dSOMAP: Clustering and Classificationof Remotely Sensed Imagery by Combining Multiple Kohonen SelfOrganizing Maps and Associative Memory,” Proc. of IJCNN’93, vol. III,
pp. 2464-2467, 1993.
[4] J. Vesanto and E. Alhoniemi, “Clustering of the Self-Organizing Map,”
IEEE Trans. Neural Networks, vol. 11, no. 3, pp. 586–600, 2002.
[5] C. Derek and E. Adrian, “Finding Curvilinear Features in Spatial Point
Patterns: Principal Curve Clustering with Noise,” IEEE Transactions on
Pattern Analysis and Machine Intelligence, vol. 22, no. 6, pp. 601-609,
2000.
[6] P. Doucette, P. Agouris and A. Stefanidis, “Self-Organized Clustering forRoad Extraction in Classified Imagery,” ISPRS Journal of Photogrammetry and Remote Sensing, vol. 55, Issues 5-6, pp. 347-358, 2001.
[7] A. Ultsch, “Clustering with SOM: U*C,” Proc. Workshop on SelfOrganizing Maps., pp.75-82, 2005.
[8] H. Matsushita and Y. Nishio, “Tentacled Self-Organizing Map for Effective Data Extraction,” Proc. International Neural Network Conference on
Neural Networks, pp. 1929-1936, 2006.
[9] A. Ultsch and H. P. Siemon, “Kohonen’s Self Organizing Feature Maps
for Exploratory Data Analysis,” Proc. International Neural Network
Conference, pp. 305-308, 1990.
[10] X. Zhang and Y. Li, “Self-organizing map as a new method for clusteringand data analysis,” Proc. International Neural Network Conference on
Neural Networks, pp. 2448-2451, 1993.
[11] M. A. Kraaijveld, J. Mao, and A. K. Jain, “A nonlinear projection method based on Kohonen’s topology preserving maps,” IEEETrans. Neural Netws., vol. 6, no. 3, pp. 548-559, 1995.
[12] N. R. Pal and V. K. Eluri, “Two efficient connectionist schemes
for structure preserving dimensionality reduction,” IEEE Trans. NeuralNetws., vol. 9, no. 6, pp. 1142-1154, 1998.
[13] M. C. Su and H. T. Chang, “A new model of self-organizing neuralnetworks and its application in data projection,” IEEE Trans. NeuralNetws., vol. 12, no. 1, pp. 153-158, 2001.
[14] T. Haraguchi, H. Matsushita and Y. Nishio, “Community Self-- 101 -Organizing Map and its Application to Data Extraction,” Proc. International Neural Network Conference on Neural Networks, June. 2009.