2006 RISP International Workshop
on Nonlinear Circuits and Signal Processing (NCSP'06)
Waikiki Beach Marriott, Honolulu,Hawaii, USA, March 3-5, 2006

Data Extraction by Self-Organizing Map Containing Neurons with Additional States
Masato Tomita† , Haruna Matsushita† and Yoshifumi Nishio††Tokushima University
2-1 Minami-Josanjima, Tokushima, Japan
Phone:+81-88-656-7470, FAX:+81-88-656-7471
Email: {tomita, haruna, nishio}@ee.tokushima-u.ac.jp

Abstract

update of the weight vectors. Furthermore, as decreasing thevalue, the updated rate of neurons reduces. From the feature,
proposed SOM tends to self-organize the area where the input
data are concentrated.
In the Section 2, the algorithm of the conventional SOM
is introduced. In the Section 3, we explain the learning algorithm of the proposed SOM algorithm in detail. In the Section
4, we investigate the behavior of the proposed SOM with anapplication to clustering input data including a lot of noises.
We can see that the proposed SOM successfully extracts thecluster.

The Self-Organizing Map (SOM) is unsupervised neural network introduced by Teuvo Kohonen. SOM attracts attentions
for clustering in these years. In this study, we propose SOM
containing neurons with additional states. Basic algorithm ofproposed SOM is the same as conventional SOM. Each neuron of the proposed SOM has an additional state value. The
value decreases according to the update of the weight vectors. Furthermore, as decreasing the value, the updated rateof neurons reduces. The proposed SOM self-organizes only
cluster from the input data which include a lot of noises. The
behavior of proposed SOM is investigated with an applicationto clustering input data including a lot of noises. We can see 2. Self-Organizing Map (SOM)that the proposed SOM successfully extracts the cluster.
In this Section, we explain about the algorithm of the conventional SOM.
1. Introduction
SOM has two-layer structure of the input layer and thecompetitivelayer. In the competitive layer, m neurons are
In data mining, clustering is one of the most importantarrangedasa
regular 2-dimensional grid. The range of theissues and is also very useful for many applications, suchelementsofd-dimensionalinput data xj = (xj1 , xj2 , · · ·, xjd )as industrial engineering, image processing, biology and(j=1,2,···,N)
areassumedto be from 0 to 1.
medicine. Then, the Self-Organizing Map (SOM) attracts(SOM1)
The
initialvaluesof all the weight vectors
attentions for clustering in recent years. SOM is unsuperw=(w,w,···,w)(i=1, 2, · · · , m) are given bei
i1i2
id
vised neural network introduced by Kohonen in 1982 [1] andtween
0and1atrandom.
is a model simplifying self-organization process of the brain.
SOM maps multidimensional data onto a 2-dimensional grid. (SOM2) An input vector xj is inputted to all the neurons at
SOM can classify the input data according to similarities and the same time in parallel.
patterns which are obtained by the distance between neurons (SOM3) We find the winner neuron c(j) by calculating theand is applied to wide fields of data classifications. Although distance between the input vector xj and the weight vector
many methods in order to extract clusters by using SOM have wi of neuron i, according to;
been proposed [2]-[8], it seems to be very difficult to conc(j) = arg min{kwi − xj k}.
(1)
struct a simple method using SOM for universal input data.
i
On the one hand, in the world, the amount and complexity ofdata increase from year to year. Therefore, it is important to In other words, the winner neuron c(j) is the neuron with theinvestigate various extraction methods of clusters from data weight vector nearest to the input vector xj . In this study,including a lot of noises.
Euclidean distance is used for Eq. (1).
In this study, we propose SOM containing neurons with ad- (SOM4) The weight vectors of the neurons are updated as;
ditional states. The proposed SOM self-organizes only clusterfrom the input data including a lot of noises. The most imporwi (t + 1) = wi (t) + hc(j),i (t)(xj − wi (t)).
(2)
tant feature of the proposed SOM is that each neuron has anadditional state value. The value decreases according to the where t is the learning step. hc(j),i (t) is called the neighbor-- 29 -

hood function and it is described as follows;µ¶kr i − r c(j) k2hc(j),i (t) = α(t) exp −2σ 2 (t)The initial value of αp and σp are chosen as follows;
αp (0) = 0.05, σp (0) = 10.0.

(3)

By virtue of this additional state, SOM is updated as follows.
In the initial stage of the program, neuron i converges
where α(t) is the learning rate, r i and r c(j) are the vectoat
oncein the area of cluster. Furthermore, in the last stage
rial locations on the display grid, and σ(t) corresponds to theof
theprogram,neuron i does not move well to spread out ofwidth of the neighborhood function. α(t) and σ(t) decrease
the cluster. Consequently we can gather neurons in the areawith time according to the following equations;of the cluster.
α(t) = α(0) (1 − t/T ) ,(4)σ(t) = σ(0) (1 − t/T ) .
4. Simulation Results
where T is the maximum number of the learning.
4.1. 2-dimensional input data(SOM5) The steps from (SOM2) to (SOM4) are repeated for
all the input data.
Input data is 2-dimensional random data of 400 points
whose distribution is non-uniform as Fig. 1(a). 200 points
are distributed within a small range from 0.3 to 0.5 horizontally and from 0.3 to 0.5 vertically. The remaining 200 points
3. SOM Containing Neurons with Additional States
are uniformly distributed between 0 and 1 at random. The
In this study, we propose SOM containing neurons with conventional SOM and the proposed SOM have 100 neurons
additional states.
(10 × 10). The parameters for the learning are chosen as folWe assign a variable pi (i = 1, 2, · · ·, m) to each neuron i lows;of SOM. This additional state pi is updated according to theα(0) = 0.7, σ(0) = 3.0.
following rule and also influences the update of the weight
We repeat the learning 5 times for all input data. The simuvectors.
lation result of the conventional SOM is shown in Fig. 1(b).
pi (t + 1) = pi (t) − 0.5hc(j),i (t)(xj − wi (t)).
(5) Fig. 2 shows the learning process of the proposed SOM indetail. We can see that the conventional SOM self-organizes
all the input data including a lot of noises. On the other hand,
We set the initial value of pi as 3.0.
the proposed SOM self-organizes only the cluster and is not(Case 1) For 2 < pi ≤ 3:
The weight vector of neuron i is updated according to Eq. (2). influenced by noises.
(Case 2) For 1 < pi ≤ 2:
The weight vector of neuron i is updated according to;
1
1
0.9

0.9

0.8

0.8

(Case 3) For 0 < pi ≤ 1:
The weight vector of neuron i is updated according to;0.7

0.7

0.6

0.6

0.5

0.5

wi (t + 1) = wi (t) + 0.1hc(j),i (t)(xj − wi (t)).

0.4

0.4

0.3

0.3

0.2

0.2

wi (t + 1) = wi (t) + 0.5hc(j),i (t)(xj − wi (t)).

(6)

(7)

(Case 4) For pi ≤ 0:
pi does not decrease anymore. The weight vector of neuron iis updated according to;

0.100

wi (t + 1) = wi (t) + 0.1hpc(j),i (t)(xj − wi (t)).

0.1
0.2

0.4

0.6

(a)

0.8

1

00

0.2

0.4

0.60.81(b)(8) Figure 1: Clustering of 2-dimensional input data. (a) Input
data. (b) Learning results of conventional SOM.
where hpc(j),i (t) is the neighborhood function described asfollows;
The concept using the proposed SOM can be exploited toµ¶
kr i − r c(j) k2
extract the data only in clusters of the input data including
hpc(j),i (t) = αp (t) exp −.
(9)2σp2 (t)a lot of noises, because proposed SOM can find such areasby themselves. We carry out the extraction of cluster afterαp (t) = αp (0) (1 − t/T ) ,(10) Fig. 1(b) and Fig. 2(f). The extraction method is a relatively
simple as follows. After learning, the input data which isσp (t) = σp (0) (1 − t/T ) .

- 30 -

1

1

1

0.9

0.9

0.9

0.8

0.8

0.8

0.7

0.7

0.7

0.6

0.6

0.6

0.5

0.5

0.5

0.4

0.4

0.4

0.3

0.3

0.3

0.2

0.2

0.2

0.1

0.1

00

0.2

0.4

0.6

0.8

1

00

0.1
0.2

0.4

(a)

0.6

0.8

1001

1

1

0.90.9

0.8

0.8

0.8

0.7

0.7

0.7

0.6

0.6

0.6

0.5

0.5

0.5

0.4

0.4

0.4

0.3

0.3

0.3

0.2

0.2

0.2

0.1

0.1

0.1

00

000.4

0.6

(d)

0.8

1

0.2

0.4

0.4

0.6

0.8

1

0.6

0.8

1(c)

0.9

0.2

0.2

(b)

0.6

(e)

0.8

1

00

0.2

0.4

(f)

Figure 2: Learning process of proposed SOM. (a) Initial state (t = 0). (b) t = 20. (c) t = 50. (d) t = 100. (e) t = 200.
(f) Learning results of proposed SOM (t = 400).

within a radius of R from any neurons on the map are classi- cially as follows. Total number of the input data is 400 (N =fied into the cluster.
400). 200 points are distributed in the cluster whose values
are distributed according to x-coordinate : N (0.2, 0.0852 ), y1
1coordinate : N (0.3, 0.092 ), and z-coordinate : N (0.7, 0.092 ).
0.9
0.9
The remaining 200 points of the input data are distributed be0.8
0.8
tween 0 and 1 at random.
0.7
0.7
0.6
0.6
The conventional SOM and the proposed SOM have 100
0.5
0.5
neurons (10×10). The parameters for the learning are chosen
0.4
0.4
as follows;
0.3
0.3
α(0) = 0.7, σ(0) = 3.0.
0.2

0.2

We repeat the learning 5 times for all input data.
Figures 4(b) and (c) show the learning results of the con0.2
0.4
0.6
0.8
1
0.2
0.4
0.6
0.8
1
ventional
SOM and the proposed SOM, respectively (R =(a)(b)0.1). We can see that SOM containing neurons with addiFigure 3: Extraction of cluster. (a) Cluster extracted by contional states self-organizes only the cluster. Figure 5(b) shows
ventional SOM. (b) Cluster extracted by proposed SOM.
the extracted cluster by the proposed SOM method. We canconfirm that the noises are removed and only cluster part can
The simulation results of the conventional SOM and the be extracted very well.
proposed SOM are shown in Figs. 3(a) and (b), respectively(R = 0.05). As we can see from figures, the cluster obtained 5. Conclusions
by the conventional SOM includes a lot of noises. However,In this study, we have proposed the neuron with additional
the proposed SOM can successfully extract only the cluster.
states of SOM. We have explained the differences between
SOM and proposed SOM with learning algorithm and have
4.2. 3-dimensional input data
investigated its behavior caused by the additional states. FurFurthermore, we carry out simulation for 3-dimensional in- thermore, we have applied proposed SOM to extract clusters
put data shown in Fig. 4. The input data is generated artifi- including a lot of noises and have confirmed the efficiency.
0.1

0.1

0000- 31 -11

1

0.8

0.8

0.8

0.6

0.6

0.6

0.4

0.4

0.4

0.2

0.2

0.2

010101

10.5

10.5

0.5
0 0

0.50 0(a)

10.5

0.5
0 0(b)(c)Figure 4: Clustering of 3-dimensional input data. (a) Input data. (b) Learning results of conventional SOM. (c) Learning
results of proposed SOM.

11

0.8

0.8

0.6

0.6

0.4

0.4

0.2

0.2

01

[5] C. Derek and E. Adrian, “Finding Curvilinear Featuresin Spatial Point Patterns: Principal Curve Clustering withnoise.” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 6, pp. 601-609, 2000.

0110.510.50.5[6] P. Doucette, P. Agouris and A. Stefanidis, “SelfOrganized Clustering for Road Extraction in ClassifiedImagery,” ISPRS Journal of Photogrammetry and RemoteSensing, vol. 55, pp. 347-358, 2001.

0.5[7] M. Gervey and S. Lek, “Patterning and Clustering Ecological Assemblages,” Proc. of WSOM’05, pp. 9-16,
2005.
Figure 5: Extraction of cluster. (a) Cluster extracted by conventional SOM. (b) Cluster extracted by proposed SOM.
[8] H. Matsushita and Y. Nishio, “Competing Behavior ofTwo Kinds of SOMs and its Application to Clustering,”
Proc. of WSOM’05, pp. 355-362, 2005.
In the future, we try to discover a new way to more extract
data than proposed SOM. And we try to utilize it in diverse
fields such as sound data processing.
0 00 0(a)(b)

References[1] T. Kohonen, Self-Organizing Maps, Berlin, Springer,
vol. 30, 1995.
[2] Y. Cheng, “Clustering with Competing Self-Organizing
Maps,” Proc. of IJCNN’92, vol. IV, pp. 785-790, 1992.
[3] W. Wan and D. Fraser, “M2dSOMAP: Clustering andClassification of Remotely Sensed Imagery by Combining Multiple Kohonen Self-Organizing Maps and Associative Memory,” Proc. of IJCNN’93, vol. III, pp. 24642467, 1993.
[4] J. Vesanto and E. Alhoniemi, “Clustering of the SelfOrganizing Map,” IEEE Transactions on Neural Networks, vol. 11, no. 3, pp. 586-600, 2000.

- 32 -