2007 RISP International Workshop on Nonlinear Circuits and Signal Processing (NCSP'07)
Shanghai Jiao Tong University, Shanghai, China, Mar. 3-6, 2007

Disconnecting Self-Organizing Map
Haruna MATSUSHITA and Yoshifumi NISHIO
Tokushima University
2-1 Minami-Josanjima, Tokushima 770-8506, Japan
Phone:+81-88-656-7470 FAX:+81-88-656-7470
Email: {haruna, nishio}@ee.tokushima-u.ac.jp

Abstract
The Self-Organizing Map (SOM) has problems with some inactive neurons which have affected a result of clustering. In
this study, we propose a new SOM algorithm which is theDisconnecting Self-Organizing Map (DSOM). In the initialstate, all neurons of DSOM are directly or indirectly connected each other. However, connections between neurons
located at distant locations are cut with learning. We canconfirm that the result of using DSOM includes no inactive
neurons and DSOM can obtain a more effective map reflecting the distribution state of input data than the conventionalSOM.

learning. The neuron is disconnected from the winner if theconnection strength between them becomes zero.
In Section II, we explain the learning algorithm of theconventional SOM. In Section III, the learning algorithm ofDSOM in detail. The learning process and the behaviors ofDSOM are investigated in Section IV with an application toclustering of 2-dimensional input data. The clustering ability
is evaluated by the visualization of the result. We can confirm
that the result of using DSOM includes no inactive neurons.
Furthermore, in Section V, we apply DSOM to feature extraction. We can see that DSOM can obtain a more effective map
reflecting the distribution state of input data than the conventional SOM.

1. Introduction

2. Self-Organizing Map (SOM)

In recent years, the Self-Organizing Map (SOM) is widelyused in classification and feature extraction tasks [1]-[5].
SOM is one of the unsupervised neural networks introduced
by Kohonen in 1982 [6] and is a model simplifying selforganization process of the brain. In the learning algorithmof SOM, a winner, which is a neuron with the weight vector
closest to the input vector, and its neighboring neurons are updated, regardless of the distance between the input vector andthe neighboring neurons. For this reason, if we apply SOM toclustering of input data which includes some clusters locatedat distant locations, there are some inactive neurons between
the clusters. Because there are inactive neurons on an area
without the input data, we are misled into thinking that there
are some input data between clusters.
In this study, we propose a new SOM algorithm which is
the Disconnecting Self-Organizing Map (DSOM). In the initial state, all neurons of DSOM are directly or indirectly connected each other. However, connections between neurons
located at distant locations are cut with learning. We explain
contours of the learning algorithm of DSOM. Each connection between neurons of DSOM has a connection strength.
The connection strength between the winner and the neuron,
which is the most distant from the input data in a set of direct
topological neighbors of the winner, is decreased with each

We explain the learning algorithm of the conventional
SOM. SOM consists of m neurons located at a regular lowdimensional grid, usually a 2-D grid. The basic SOM algorithm is iterative. Each neuron i has a d-dimensional weightvector wi = (wi1 , wi2 , · · · , wid ) (i = 1, 2, · · · , m). The initial values of all the weight vectors are given over the input
space at random. The range of the elements of d-dimensional
input data xj = (xj1 , xj2 , · · · , xjd ) (j = 1, 2, · · · , N ) are
assumed to be from 0 to 1.
(SOM1) An input vector xj is inputted to all the neurons at
the same time in parallel.
(SOM2) Distances between xj and all the weight vectors arecalculated. The winner, denoted by c, is the neuron with theweight vector closest to the input vector xj ;c = arg min{kwi − xj k},i(1)where k · k is the distance measure, in this study, Euclideandistance.
(SOM3) The weight vectors of the neurons are updated as;
wi (t + 1) = wi (t) + hc,i (t)(xj − wi (t)),(2)

where t is the learning step. hc,i (t) is called the neighborhood- 425 -

function and is described as a Gaussian function;¶µkr i − r c k2
hc,i (t) = α(t) exp −,2σ 2 (t)(3)

1234567891021011 12 13 14 15
where kr i −r c k is the distance between map nodes c and i on
the map grid, α(t) is the learning rate, and σ(t) corresponds
16 17 18 19 20
to the width of the neighborhood function. Both α(t) andσ(t) decrease with time as follows;
21 22 23 24 25´t/T³)(a)(b),α(t) = α(0) α(Tα(0)(4) Figure 1: Neighborhood distances of the rectangular grid.
³´t/T)σ(t) = σ(0) σ(T,(a) Neighborhood distances nc,i of c = 13. nc,14 = 1,σ(0)nc,12 = 3, nc,5 = nc,22 = m (namely, 25). (b) Discrete
where T is the maximum number of the learning.
neighborhoods (size 0, 1 and 2) of the centermost neuron(SOM4) The steps from (SOM1) to (SOM3) are repeated for when all the neurons are connected.
all the input data.

The connection strength between c and p is set to initial value(this means “refresh” the connection reference);

3. Disconnecting SOM (DSOM)
We explain a proposed new SOM algorithm, DSOM. The
initial state of all neurons of DSOM are directly or indirectlyconnected to all neuron. However, the connection between
neurons located at distant locations is cut with learning. The
connection between each neuron of DSOM has a connection
strength Cs(c,i) . The initial values of all the weight vectors
are given over the input space at random.

Cs(c,p) = Cs0 ,(8)where Cs0 is the initial value of Cs.
(DSOM5) A disconnecting neuron q is found. q is the most
distant from xj in N c1 . However, if kq is smaller than thenumber of N c1 , the disconnecting neuron q does not exist,and we perform (DSOM7).

(DSOM1) An input vector xj is inputted to all the neurons at
q = arg max{kwi − xj k}, i ∈ N c1 .
(9)
the same time in parallel.
i(DSOM2) Distances between xj and all the weight vectors
are calculated, and the rank order of distances, denoted by The connection strength between c and q is decreased;k = 0, · · · , m − 1, is calculated. ki is the rank of wi , namely,Cs(c,q) = Cs(c,q) − kwq − xj k2 .
(10)
kc = 0 is the rank of the winner c which is closest to xj
according to Eq. (1).
(DSOM3) The weight vectors of the neurons are updated as;(DSOM6) The disconnecting neuron q is disconnected fromwi (t + 1) = wi (t) + hD c,i (t)(xj − wi (t)),(5) the winner c if its connection strength becomes smaller thanzero.
where hD c,i (t) is the neighborhood function of DSOM;(DSOM7) The steps from (DSOM1) to (DSOM6) are reµ¶peated for all the input data.
ki + nc,ihD c,i (t) = α(t) exp − 2
(6),
σ (t)
where nc,i is the neighborhood distance between c and eachneuron i. The neighborhood distances are defined as shortestpath distances between connected map nodes as Fig. 1(a). If
a neuron i is not connected directory or indirectly to c, nc,i isequal to the number of neurons m.
(DSOM4) A connection between c and a neuron p which isclosest to xj in N c1 is created, if it does not exist.

4. Application to Clustering

We consider 2-dimensional input data as shown in Fig. 2(a)to clustering. The input data is generated artificially as follows. Total number of the input data N is 15000, and the input data include two clusters. 7500 data are distributed withina range from 0.2 to 0.8 horizontally and from 0.1 to 0.3 vertically. The remaining 7500 data are distributed within a range
p = arg min{kwi − xj k}, i ∈ N c1 ,(7) from 0.2 to 0.8 horizontally and from 0.7 to 0.9 vertically. All
ithe input data are sorted at random.
where N c1 is the set of direct topological neighbors of c as
Both the conventional SOM and DSOM has m = 100 neuFig. 1(b),rons (10 × 10). We repeat the learning until all input data is- 426 -

1

1

1

1

0.9

0.9

0.9

0.9

0.8

0.8

0.8

0.8

0.7

0.7

0.7

0.7

0.6

0.6

0.6

0.6

0.5

0.5

0.5

0.5

0.4

0.4

0.4

0.4

0.3

0.3

0.3

0.3

0.2

0.2

0.2

0.2

0.1

0.1

0.1

0.1

00

0
0

0
0

0.2

0.4

0.6

0.8

1

0.2

0.4

(a)

0.6

0.8

1

0.2

(b)

0.4

0.6

0.8

1

001

1

1

1

0.9

0.9

0.9

0.8

0.8

0.8

0.8

0.7

0.7

0.7

0.7

0.6

0.6

0.6

0.6

0.5

0.5

0.5

0.5

0.4

0.4

0.4

0.4

0.3

0.3

0.3

0.3

0.2

0.2

0.2

0.2

0.1

0.1

0.1

0.2

0.4

0.6

0.8

001

(e)

0.2

0.4

0.6

0.8

1

00(f)

0.4

0.6

0.8

1

0.6

0.8

1(d)

0.9

00

0.2

(c)

0.1
0.2

0.4

(g)

0.6

0.8

1

0
0

0.2

0.4

(h)

Figure 3: Learning process of DSOM. (a) Initial state (t = 0). (b) t = 1000. (c) t = 3000. (d) t = 5000. (e) t = 7500.
(f) t = 9500. (g) t = 12500. (f) Learning result (t = 15000).

110.9

0.9

0.8

0.8

0.7

0.7

0.6

0.6

0.5

0.5

The other side, the result of DSOM and its learning process
are shown in Fig. 3. We can see from Fig. 3(h) that there are
no inactive neurons between the two clusters.

Let us consider the learning process. The initial states ofweightvectors of neurons are random values, and all neurons
0.4
0.4
are
directly
or indirectly connected each other as Fig. 3(a).
0.30.3In the early-stage of the learning as Figs. 3(b) and (c), all
0.2
0.2
0.1
0.1
neurons are still connected mutually and all neurons are self00organizing all input data. Furthermore, from Fig. 3(d), there
00.2
0.4
0.6
0.8
1
0
0.2
0.4
0.6
0.81
are some inactive neurons because the neurons located at dis(a)(b)
tant locations are not disconnected. In the middle stage asFigure 2: Clustering of 2-dimensional input data. (a) Input Figs. 3(e) and (f), we can see that the connections between
data. (b) Learning result of the conventional SOM.
neurons of the two clusters are cut, and there are no inactive neurons. This is because the neurons of DSOM are notaffected by neurons which are self-organizing another clusinputted, namely T = 15000. The parameters of the learning ter, so, the neurons can learn more distant for the distant
input data, than the conventional SOM. In the last stage asare chosen as follows;
Figs. 3(g) and (h), we can confirm that DSOM are completely(For SOM)
separated by two and each neuron group self-organizes eachcluster.
α(0) = 0.5, α(T ) = 0, σ(0) = 5, σ(T ) = 0.
Figure 4 shows distances between adjacent neurons oflearning results of Figs. 2(b) and 3(h). This figure thus viα(0) = 0.5, α(T ) = 0.05, σ(0) = 40, σ(T ) = 0.5, Cs0 = 12. sualizes the cluster structure of the map. Black circles on thisfigure mean large distance between neighboring map nodes.
The learning result of the conventional SOM is shown in Clusters are typically uniform areas of white circles. We canFig. 2(b). We can see that there are some inactive neurons see that the boundary line of DSOM is clearer than the conbetween the two clusters.
ventional SOM because DSOM has no inactive neurons.
(For DSOM)- 427 -

1

1

1

0.9

0.9

0.9

0.8

0.8

0.8

0.7

0.7

0.7

0.6

0.6

0.6

0.5

0.5

0.5

0.4

0.4

0.4

0.3

0.3

0.3

0.2

0.2

0.2

0.1

0.1

0.1

00

0
0

0.2

0.4

0.6

(a)

0.8

1

0.2

0.4

0.6

(b)

0.8

1

0
0

0.2

0.4

0.6

0.8

1(c)

Figure 5: Feature extraction for doughnut-shaped data. (a) Input data. (b) Learning result of conventional SOM. (c) Learning
result of DSOM.

tial state, all neurons of DSOM are directly or indirectly connected each other. However, connections between neurons
located at distant locations are cut with learning. Each connection between neurons of DSOM has a connection strength.
The connection strength between the winner and the neuron,
which is the most distant from the input data in a set of direct
topological neighbors of the winner, is decreased with eachlearning. The neuron is disconnected from the winner if the(a)(b)
connection strength between them becomes zero.
The learning process and the behaviors of DSOM were inFigure 4: Visualization of learning result. (a) Conventional
vestigatedwith applications to clustering of 2-dimensional inSOM. (b) DSOM.
put data and feature extraction. We confirmed that the resultof using DSOM includes no inactive neuron and DSOM canobtain the more effective map reflecting the distribution state
5. Application to Feature Extractionof input data than the conventional SOM.
Furthermore, we apply DSOM to feature extraction. Weuse the “doughnut”-shaped data shown as Fig. 5 for the input
data. Total number of the input data N is 2000. All the input
data are sorted at random.
We repeat the learning 7 times for all input data, namely
T = 14000. We use the same parameters as for Fig. 2(a)except Cs0 = 11.5 for DSOM.
Figures 5(b) and (c) show the learning results of the conventional SOM and DSOM, respectively. From the learning
result of SOM, we can see that there are some inactive neurons in the doughnut hole. However, there are no inactive
neurons in the result DSOM. This is because neurons locatedat distant locations is disconnected, so, the neurons can learn
more distant for the distant input data. Therefore, DSOM canobtain the more effective map reflecting the distribution stateof input data, than the conventional SOM.

References[1] Y. Cheng, “Clustering with Competing Self-Organizing
Maps,” Proc. of IJCNN’92, vol. IV, pp. 785-790, 1992.
[2] W. Wan and D. Fraser, “M2dSOMAP:Clustering andClassification of Remotely Sensed Imagery by Combining Multiple Kohonen Self-Organizing Maps and Associative Memory,” Proc. of IJCNN’93, vol. III, pp. 24642467, 1993.
[3] J. Vesanto and E. Alhoniemi, “Clustering of the SelfOrganizing Map,” IEEE Transactions on Neural Networks, vol. 11, no. 3, pp. 586-600, 2002.
[4] I. Lapidot, H. Guterman and A. Cohen, “UnsupervisedSpeaker Recognition Based on Competition Between
Self-Organizing Maps,” IEEE Transactions on NeuralNetworks, vol. 13, no. 4, pp. 877-887, 2002.
[5] H. Matsushita and Y. Nishio, “Reunifying SelfOrganizing Map,” Proc. of NOLTA’06, pp. 207-210,
6. Conclusions
2006.
[6] T. Kohonen, Self-organizing Maps, Springer, Berlin,
In this study, we proposed a new SOM algorithm which is1995.
the Disconnecting Self-Organizing Map (DSOM). In the ini-- 428 -