NOLTA 2006
11-14 september
Bologna - ITALY

Reunifying Self-Organizing Map
Haruna Matsushita† and Yoshifumi Nishio††Department of Electrical and Electronic Engineering, Tokushima University
Email: {haruna, nishio}@ee.tokushima-u.ac.jp
Abstract— The Self-Organizing Map (SOM) attracts
attentions for clustering in these years. In this study, we
propose a Reunifying Self-Organizing Map which is a new
SOM algorithm. The initial state of all neurons of the
proposed SOM are connected to no neuron. However, theneurons are connected gradually to other neurons as learning progressed. The behavior of the reunifying SOM isinvestigated with application to clustering. We can confirm that the result of using the reunifying SOM includes
no inactive neuron.
1. Introduction
Since we can accumulate a huge amount of datain these years, it is important to investigate various
clustering methods. Then, the Self-Organizing Map(SOM) attracts attentions in recent years. SOM is anunsupervised neural network introduced by Kohonen
in 1982 [1] and is a model simplifying self-organization
process of the brain. SOM obtains statistical feature ofinput data and is applied to a wide field of data classifications [2]-[5]. We can obtain the map reflecting
the distribution state of input data using SOM. However, if we apply SOM to clustering of the input data
which includes some clusters located at distant locations, there are some inactive neurons between clusters. Because inactive neurons are on a part without
the input data, we are misled into thinking that there
are some input data between clusters.
In this study, we propose a new SOM algorithm
which is the Reunifying Self-Organizing Map. Theinitial state of all neurons of the proposed SOM areconnected to no neuron. However, each neuron has its
own physical location on the 2-dimensional grid, so,
the neurons are connected gradually to other neurons
as learning progressed: hence, the name “reunifying”.
In Section II, we explain the learning algorithm of
the conventional SOM. In Section III, the learning algorithm of the reunifying SOM in detail. The learning behaviors of the reunifying SOM are investigatedin Section IV with applications to clustering of various 2-dimensional input data. Furthermore, clustering
ability is evaluated by visualization of results. We cansee that there are no inactive neurons using the reunifying SOM.
2. Self-Organizing Map (SOM)
We explain the learning algorithm of the conventional SOM. SOM consists of m neurons located at aregular low-dimensional grid, usually a 2-D grid. The
basic SOM algorithm is iterative. Each neuron i has

a d-dimensional weight vector wi = (wi1 , wi2 , · · · , wid )(i = 1, 2, · · · , m). The initial values of all the weightvectors are given over the input space at random.
The range of the elements of d-dimensional input dataxj = (xj1 , xj2 , · · · , xjd ) (j = 1, 2, · · · , N ) are assumed
to be from 0 to 1.
(SOM1) An input vector xj is inputted to all theneurons at the same time in parallel.
(SOM2) Distances between xj and all the weight vectors are calculated. The winner, denoted by c, is theneuron with the weight vector closest to the input vector xj ;c = arg min{kwi − xj k},(1)iwhere k · k is the Euclidean distance.
(SOM3) The weight vectors of the neurons are updated as;
wi (t + 1) = wi (t) + hc,i (t)(xj − wi (t)),(2)

where t is the learning step. hc,i (t) is called the neighborhood function and is described as a Gaussian function;µ¶kr i − r c k2
hc,i (t) = α(t) exp −,(3)
2σ 2 (t)where kr i − r c k is the distance between map nodesc and i on the map grid, α(t) is the learning rate,and σ(t) corresponds to the width of the neighborhoodfunction. Both α(t) and σ(t) decrease monotonicallywith time as follows;α(t) = α(0)(1 − t/T ), σ(t) = σ(0)(1 − t/T ),(4)

where T is the maximum number of the learning.
(SOM4) The steps from (SOM1) to (SOM3) are repeated for all the input data.
3. Reunifying Self-Organizing Map
In this study, we propose a new algorithm of SOM,
Reunifying Self-Organizing Map. The initial state of
all neurons of the reunifying SOM are connected to noneuron, but each neurons have its own physical location on the 2-D grid. The initial value of the weightvectors are given at orderly position.
(RSOM1) An input data xj is inputted to all theneurons at the same time in parallel.
(RSOM2) The winner c is found according to Eq. (1).
(RSOM3) The neighborhood distance between c andeach neuron i, denoted by nc,i , is calculated. The207

(a)
(b)
Figure 1: Neighborhood distances of the rectangulargrid. (a) Discrete neighborhoods (size 0, 1 and 2) ofthe centermost neuron when all the neurons are connected. (b) Neighborhood distances nc,i of c = 13.
nc,14 = 1, nc,12 = 3, nc,5 = nc,22 = m (namely, 25).

(a)(b)(c)Figure 2: Example of connection of neurons. c = 13.
N c1 = [8, 12, 14, 18] (a) Sq = 12. q = 12. (b) Sq =[14, 18]. q is the neuron with the weight vector
smaller distance between the input vector. (c) Sq =[8, 12, 14, 18] = N c1 . q is not chosen.

4. Application to Clustering
neighborhood distances are defined as shortest-path
distances between connected map nodes as Fig. 1. If aneuron i is not connected directly or indirectly to thewinner c, nc,i is equal to the number of neurons m.
(RSOM4) The weight vectors of the neurons are updated as;
wi (t + 1) = wi (t) + hRc,i (t)(xj − wi (t)),(5)

where hRc,i (t) is the neighborhood function of the reunifying SOM and described as;¶µ(nc,i /m + kwi − xj k)2hRc,i (t) = α(t) exp −. (6)2σ 2 (t)(RSOM5) A set of 1-neighborhood neurons of winner c, on the assumption that all the neurons are connected (as Fig. 1(a)), is denoted as N c1 . The numberof N c1 is between two and four when the neurons arelocated on the 2-D rectangular grid.
A set of the neurons, whose neighborhood distanceis the longest in N c1 , are denoted as Sq (as Fig. 2).
Sq = arg max{nc,i }, i ∈ N c1 .
i(7)If nc,Sq = 1 (as Fig. 2(c)), we perform (RSOM8).
However, if not, we perform (RSOM6).
(RSOM6) The connecting neuron q is chosen fromSq, according to;q = arg min{kwi − xj k}, i ∈ Sq.
i4.1. Simulation 1First, we consider 2-dimensional input data as
shown in Fig. 3(a). The input data is generated artificially as follows. Total number of the input data N
is 600, and the input data include two clusters. 300
data are distributed within a range from 0.2 to 0.8 horizontally and from 0.1 to 0.3 vertically. The remaining
300 data are distributed within a range from 0.2 to0.8 horizontally and from 0.7 to 0.9 vertically. All theinput data are sorted by random.
Both the conventional SOM and the reunifying SOM
has m = 100 neurons (10×10). We repeat the learning
20 times for all input data, namely T = 12000. The
parameters of the learning are chosen as follows;(For SOM)
α(0) = 0.5, σ(0) = 3,
(For Reunifying SOM)
α(0) = 0.5, σ(0) = 0.8, Dmin =(RSOM7) The winner c is directly connected to aconnecting neuron q, namely, nc,q becomes 1.
(RSOM8) The steps is returned to (RSOM1) and isrepeated for all the input data.

2/10, Dmax =√2/2.

The simulation result of the conventional SOM isshown in Fig. 3(b). We can see that there are some
inactive neurons between two clusters.
The other side, the result of the reunifying SOM
and its learning process are shown in Fig. 4. We cansee from Fig. 4(h) that there are no inactive neurons
between two clusters.

(8)

In other words, the connecting neuron q is the neuron
with the weight vector closest to xj in Sq.
If kwq − wc k ≤ D(t), we perform (RSOM7). However, if not, we except the neuron q from N c1 , andperform the simulation again from (RSOM5).
D(t) increases monotonically with time according
to;Dmax
D(t) =
t + Dmin .
(9)
T

√

1

1

0.8

0.8

0.6

0.6

0.4

0.4

0.2

0.2

00

0.2

0.4

0.6

0.8

10
0

0.2

0.4

0.6

0.8

1

(a)
(b)
Figure 3: Clustering of 2-dimensional input data.
(a) Input data. (b) Simulation result of the conventional SOM.

208

1

1

1

1

0.8

0.8

0.8

0.8

0.6

0.6

0.6

0.6

0.4

0.4

0.4

0.4

0.2

0.2

0.2

0.2

00

0.2

0.4

0.6

0.8

0
01

0.2

0.4

(a)

0.6

0.8

00

1

0.2

(b)

0.4

0.6

0.8

001

11

1

0.8

0.8

0.8

0.8

0.6

0.6

0.6

0.6

0.4

0.4

0.4

0.4

0.2

0.2

0.2

0.2

0.2

0.4

0.6

0.8

00

1

0.2

0.4

0.6

0.8

00

1

0.2

0.4

0.4

0.6

0.8

1

0.6

0.8

1(d)

1

00

0.2

(c)

0.6

0.8

00

1

0.2

0.4

(e)
(f)
(g)
(h)
Figure 4: Clustering using the reunifying SOM. (a) Initial state (t = 0). (b) t = 100. (c) t = 500. (d) t = 1700.
(e) t = 6000. (f) t = 6200. (g) t = 6700. (f) Simulation result (t = 12000).

Let us consider the learning process. The initialstate of neurons are not connected to other neurons
and are located at orderly position as Fig. 4(a). In theearly-stage of learning as Figs. 4(b) and (c), we can seethat some neurons are connected gradually to otherneurons. However, there are some groups of neurons,
because the connected neuron is preferentially-selectedfrom the neuron which is connected to no neuron. Furthermore, as Fig. 4(d), there are no neurons which areconnected to no neuron, and each two group of neurons
self-organize respective two cluster without connecting each other. In the middle stage as Fig. 4(f), twogroups of neurons are connected each other according to increase D(t). However, neurons of each groupself-organize the respective two clusters without being
mingled as Fig. 4(g). in other words, the neurons ofeach group are not influenced by each other. This isbecause the learning rate α(t) and the neighborhood
radius σ(t) decrease according to Eq. (4), namely α(t)and σ(t) are small value.

11

0.8

0.8

0.6

0.6

0.4

0.4

0.2

0.2

0000Figure 5 shows distances between neighboring neurons and thus visualizes the cluster structure of themap. Black circles on this figure mean large distance
between neighboring map nodes. Clusters are typically uniform areas of white circles. We can see thatthe boundary line of the reunifying SOM is clear thanthe conventional SOM.
4.2. Simulation 2
Next, we carry out simulation for another 2dimensional input data example shown in Fig. 6. The
input data include two clusters. Total number of input data N is 600. We repeat the learning 20 times
for all input data. The parameters of the learning are
the same√value used in the simulation 1 except forDmax = 2.
The results of SOM and the reunifying SOM are
shown in Figs. 7(a) and (b), respectively. We can seethat there are no inactive neurons in the reunifying
SOM although there are some inactive neurons in conventional SOM.
1

0.8

0.6

0.4
0.2

0.4

0.6

0.8

1

0.2

0.4

0.6

0.8

1(a)
(b)
Figure 5: Visualization of result. (a) Conventional
SOM. (b) Reunifying SOM.

209

0.2

000.2

0.4

0.6

0.8

1

Figure 6: 2-dimensional input data.

1

1

1

1

0.8

0.8

0.8

0.8

0.6

0.6

0.6

0.6

0.4

0.4

0.4

0.4

0.2

0.2

0.2

0.2

00

0.2

0.4

0.6

0.8

1

00

0.2

(a)

0.4

0.6

0.8

00

1

0.2

0.4

(b)

0.6

0.8

1

001

1

1

0.8

0.8

0.8

0.8

0.6

0.6

0.6

0.6

0.4

0.4

0.4

0.4

0.2

0.2

0.2

0.2

0.2

0.4

0.6

0.8

1000.2

0.4

0.6

0.8

001(c)(d)Figure 7: Results of clustering. (a) Conventional
SOM. (b) Reunifying SOM. (c) Visualization of theconventional SOM. (d) Visualization of the reunifying
SOM.

0.4

0.6

0.8

1

0.6

0.8

100

0.2

0.4

0.6

0.8

1(c)
(d)
Figure 9: Clustering results for concave input data.
(a) Conventional SOM. (b) Reunifying SOM. (c) Visualization of the conventional SOM. (d) Visualizationof the reunifying SOM.

4.3. Simulation 3

References

Furthermore, we carry out simulation for the concave input data shown in Fig. 8. Total number of
input data N is 600. We repeat the learning 20 times
for all input data The parameters of the learning are
the same values used in the simulation 1.
Figure 9 shows the result of SOM and the reunifying
SOM. As we can see from these figures, the clustering
ability of using the reunifying SOM method is effective.
1[1] T. Kohonen, Self-organizing Maps, Berlin, Springer,
vol. 30, 1995.
[2] Y. Cheng, “Clustering with Competing Self-Organizing
Maps,” Proc. of IJCNN, vol. IV, 1992, pp. 785-790.
[3] W. Wan and D. Fraser, “M2dSOMAP:Clustering andClassification of Remotely Sensed Imagery by Combining Multiple Kohonen Self-Organizing Maps andAssociative Memory,” Proc. of IJCNN, vol. III, 1993,
pp. 2464-2467.
[4] J. Vesanto and E. Alhoniemi, “Clustering of the SelfOrganizing Map,” IEEE Transactions on Neural Networks, vol. 11, no. 3, 2002, pp. 586-600.

0.8

0.6

[5] I. Lapidot, H. Guterman and A. Cohen, “Unsupervised
Speaker Recognition Based on Competition Between
Self-Organizing Maps,” IEEE Transactions on NeuralNetworks, 2002, vol. 13, no. 4, pp. 877-887.

0.4

0.2

000.2

0.4(b)

1000.2(a)

0.2

0.4

0.6

0.8

1

Figure 8: Concave input data.

5. Conclusions
In this study, we have proposed the new SOM algorithm which is the Reunifying SOM. The initial stateof all neurons of the reunifying SOM are connected tono neuron. However, the neurons are connected gradually to other neurons as learning progressed. We haveinvestigated its behaviors with applications to clustering, and have confirmed the efficiency.
210