2009 IEEE Workshop on
Nonlinear Circuit Networks, Tokushima, Dec 11-12, 2009

WC-SOM and its Flexible Self-Organization
Haruna Matsushita and Yoshifumi Nishio
Department of Electrical and Electronic Engineering,
Tokushima University
Email: {haruna, nishio}@ee.tokushima-u.ac.jp

Abstract— This study proposes weighted connections avoiding
false-neighbor effects for the Self-Organizing Map. The new
SOM is called WC-SOM. We investigate the effectiveness of
WC-SOM in comparison with the conventional SOM, Growing
Grid and FN-SOM. We confirm that WC-SOM enables the mostflexible self-organization among four algorithms and can obtain
the effective map reflecting the distribution state of the input
data using fewer neurons.

C1

Cm-1

nf(1,2)R1

nf(1,m+1)R2nnf(2,m+2) nf(m,2m)nnf(m+1,m+2)

nf((n-1)m+1,

Rn-1

I. I NTRODUCTION
The Self-Organizing Map (SOM) is an unsupervised neuralnetwork [1] and has attracted attention for its clustering
properties. In the learning algorithm of SOM, a winner, which
is a neuron closest to an input data, and its neighboring
neurons on the map are updated, regardless of the distance
between the input data and the neighboring neurons. For thisreason, if we apply SOM to clustering of the input data
including some clusters located at distant location, there aresome inactive neurons between clusters where without the
input data. Because the inactive neurons are on a part, we are
misled into thinking that there are some input data between
clusters. Furthermore, because the simulation time depends
on the number of neurons, it is important to utilize the usedneurons effectively by reducing the inactive neurons.
In the real world, it is not always true that neighboringhouses are physically adjacent or close to each other. In
other words, “neighbors” are not always “true neighbors”. In
addition, the relationship between neighborhoods is not fixed,
but keeps changing with time. It is important to change theneighborhood relationship flexibly according to the situation.
Meanwhile, the synaptic strength is not constant in the brain.
So far, the Growing Grid network was proposed in 1985 [2].
Growing Grid increases the neighborhood distance between
neurons by increasing the number of neurons. However, there
is not much research changing the synaptic strength.
In our past study, we proposed the SOM with FalseNeighbor degree between neurons (called FN-SOM) [4].
False-neighbor degrees (FNDs) are allocated between adjacentrows and adjacent columns of FN-SOM. FNDs act as a burdenof the distance between map nodes when the weight vectors ofneurons are updated. FN-SOM can greatly reduce the inactive
neurons, however, the algorithm has following problems. All
the FNDs between the neurons on same line are increased
simultaneously and forcibly. It often produces the increase of
FNDs between the correct-neighboring neurons, namely, it isdefined as a FALSE false-neighbor.

C2(n-1)m+2)

m

m

(a)

(b)

Fig. 1. Differences between false-neighbor degree of FN-SOM and falseneighbor weights of WC-SOM. Neurons of both SOMs are located at a n×m
rectangular grid. (a) A false-neighbor degree of row Rr (1 ≤ r ≤ n − 1)and column Ck (1 ≤ k ≤ m − 1). (b) False-neighbor weights nf (i,k) areallocated between all the directly-connected neurons i and k. k ∈ Ni 1

In this study, we incorporate the weighted connections
avoiding false-neighbor effects into SOM. The new SOM
is called SOM with Weighted Connections avoiding falseneighbor effects (WC-SOM). In WC-SOM, all the connections
between adjacent neurons are weighted to avoid false-neighboreffects unlike FN-SOM. This weights are called as falseneighbor weights (FNWs). In the algorithm of WC-SOM,
we find the winless neurons and its “false neighbors”, and
FNWs between these neurons are increased. In other words, byincreasing FNWs with learning, WC-SOM changes the neighborhood relationship more flexibly according to the situationand the shape of data.
We compare WC-SOM with the conventional SOM, Growing Grid and FN-SOM, and effectiveness of WC-SOM areinvestigated by applying to complex input data. We confirm
that WC-SOM can obtain the effective map reflecting the
distribution state of the input data using fewer neurons.
II. SOM WITH W EIGHTED C ONNECTIONS AVOIDINGFALSE - NEIGHBOR EFFECTS (WC-SOM)
The conventional method, FN-SOM, has false-neighbor
degrees (FNDs) between neurons shown as Fig. 1(a). However,
all the FNDs between neurons at same line on the map areforcibly the same value. It often produces the twist of the map.
In this study, we propose Weighted Connections avoiding
false-neighbor effects for SOM (WC-SOM). We explain WCSOM in detail in this section. WC-SOM consist of n×m neurons located at 2-dimensional rectangular grid. Each neuron i
has a d-dimensional weight vector wi = (wi1 , wi2 , · · · , wid )- 63 -

1

2

c
6
11

7
3

3
4

8
1

4
3

5

1

2

c
9
1

1
1
12
14
13
df (7,14) = 5

10
15

6
11

7
3

3
4

8
1

3

4

5

9
1

10

1
1
12
14
13
df (7,9) = 7(a)where hF c,i (t) is the neighborhood function of WC-SOM:µ¶dis(c, i)hF c,i (t) = α(t) exp −.
(4)
2σ 2 (t)15

where α(t) is the learning rate, and σ(t) corresponds tothe width of the neighborhood function. Both α(t) and σ(t)decrease with time, in this study, we use following equations;(b)Fig. 2.
The connection weight df (c,i) between the winner c and theneuron i. df (c,i) is a minimum sum-of-FNWs of neurons connected by theshortest-path distance from c to i. Values in the circle mean respective neuron
numbers. Values between connections mean FNWs nf between directlyconnected neurons. The winner c is 7. (a) Let i = 14. The possible shortestpath routes from c to i are [7 8 9 14], [7 8 13 14] and [7 12 13 14]. The
sum of FNWs of respective routes are nf (7,8) + nf (8,9) + nf (9,14) = 8,
nf (7,8) +nf (8,13) +nf (13,14) = 6 and nf (7,12) +nf (12,13) +nf (13,14) =5. Therefore, df (7,14) = 5 which is minimum value. (b) Let i = 9.
df (7,9) = 7 because the possible shortest-path route is [7 8 9] only. Notethat a route [7 12 13 14 9] is NOT adopted because it is not the shortest-pathroute although the sum of FNWs of this route is 6 which is smaller than 7.

(i = 1, 2, · · · , nm). The range of the elements of the input
data xj = (xj1 , xj2 , · · · , xjd ) (j = 1, 2, · · · , N ) are assumed
to be from 0 to 1.
In WC-SOM, a false-neighbor weight nf (i,k) (FWN) is
allocated between directly-connected neurons i and k shownas Fig. 1(b), and we propose the new neighborhood distanceconsidering FNWs. The initial value of each FNW nf between
directly-connected neurons is set to zero, and the initial valuesof all the weight vectors are given over the input space at
random. Moreover, a winning frequency γi is associated witheach neuron and is set to zero initially.
Learning Step
(Step1) Input an input vector xj to all the neurons.
(Step2) Calculate distances between xj and all the weightvectors. Find a winner, denoted by c, which is the neuron
with the weight vector closest to the input vector xj :c = arg min{kwi − xj k},i(1)where k · k is Euclidean distance measure.
(Step3) Increase the winning frequency of the winner c byγc new = γc old + 1.
(Step4) Calculate the neighboring distance dis(c, i) between
the winner c and each neuron i by considering FNWs nf as
the following measure ;dis(c, i) = kr i − r c k2 + df (c,i) ,(2)

where kr i −r c k is the Euclidean distance between map nodes cand i on the map grid. df (c,i) is the connection weight betweenc and i, and it is defined as a minimum sum-of-FNWs of theneurons connected by the shortest-path distance from c to i
(as Fig. 2).
(Step5) Update the weight vectors of all the neurons:
wi (t + 1) = wi (t) + hF c,i (t)(xj − wi (t)),(3)α(t) = α0 (1 − t/tmax ),σ(t) = σ0 (1 − t/tmax ),(5)

where α0 and σ0 are the initial values of α and σ, respectively,
and tmax isPthe maximum number of the learning.
nm(Step6) If i=1 γi ≥ λ is satisfied, find the false-neighbors
and increase the false-neighbor weights nf , according to steps
from (Step7) to (Step10). If not, perform step (Step11). In
other words, we consider the false-neighbors every time when
the learning steps are performed for λ input data.
Considering F alse-N eighbors Step
(Step7) Find a set of neurons S which have never become
the winner: S = {i | γi = 0}. If the winless neuron does notexist, namely S = ∅, return to (Step1) without considering thefalse-neighbors.
(Step8) Choose a false-neighbor fq of each neuron q in S
from the set of direct topological neighbors of q denoted as
Nq 1 . fq is the neuron whose weight vector is most distantfrom q:fq = arg max{kwi − wq k}, q ∈ S, i ∈ Nq 1 .
i(6)(Step9) Increase FNW nf (q,fq ) between each q and its falseneighbor fq as
nf (q,fq ) = nf (q,fq ) + 1,

(7)

where nf (q,fq ) = nf (fq ,q) .
(Step10) Reset the winning frequency of all the neurons tozero: γi = 0.
(Step11) Repeat the steps from (Step1) to (Step10) for all the
input data.
III. E XPERIMENTAL R ESULTS
We apply WC-SOM to complex data and compare it withthe conventional SOM, Growing Grid and FN-SOM. For theexperiment, SOM, FN-SOM and WC-SOM have nm = 100
neurons (10 × 10). Growing Grid starts learning with 2 × 2
neurons, and the maximum number of neurons is less than100. The parameters of the learning for WC-SOM are chosen
as α0 = 0.3, σ0 = 3, λ = 3000, where we use the same α0
for all the method and the same λ for FN-SOM and WC-SOM.
For the conventional SOM, Growing Grid and FN-SOM, we
use σ0 = 4.
A. For Target data set
We consider Target data set [5] which has a clustering
problem of outliers. The input data is 2-dimension and has
six clusters including 4 outliers, and the total number of the- 64 -

1

1

1

1

0.8

0.8

0.8

0.8

0.6

0.6

0.6

0.6

0.4

0.4

0.4

0.4

0.2

0.2

0.2

0.2

0

0
0

0.2

0.4

0.6

0.8

1

(a)

Fig. 3.

0
0

0.2

0.4

0.6

0.8

1

0
0

0.2

(b)

0.4

0.6

0.8

1

0

(c)

0.2

0.4

0.6

0.8

(d)

Learning results of four algorithms for Target data. (a) Conventional SOM. (b) Growing Grid. (c) FN-SOM. (d) WC-SOM.

0.16

0.2
0.14

0.14

(a)

0.2

0.12

0.12

Fig. 4.

1

0.1

0.1

0.08

0.08

0.06

0.06

0.04

0.04

0.02

0.02

0.15
0.15
0.1

0.1

0.05

0.05

(b)

(c)

(d)

U-Matrices of simulation results for Target data. (a) Conventional SOM. (b) Growing Grid. (c) FN-SOM. (d) WC-SOM.

TABLE IT WO MEASUREMENTS FOR TARGET DATA .

input data N is 770. We repeat the learning 26 times for all
the input data, namely tmax = 20020.
Learning results of the conventional SOM, Growing Gridare shown in Figs. 3(a) and (b). We can see that there are a lotof inactive neurons between clusters. The other side, from thelearned maps of FN-SOM and WC-SOM shown in Figs. 3(c)and (d), it is clear that FN-SOM and WC-SOM can greatly
reduce the number of inactive neurons in comparison with theconventional SOM.
Furthermore, in order to compare the learning performanceof WC-SOM with the other SOMs numerically, we use thefollowing well-used two measurements to evaluate the training
performance.

SOM

Growing Grid

FN-SOM

WC-SOM

Qe

0.01960.02320.01900.0175

U

0.81130.83170.92630.8917

1.8

6

1.6

5

1.4
1.2

4

1

3

0.8
0.6

2

0.4

1

0.2

0

Quantization Error Qe measures the average distance between each input vector and its winner [1]. The small value
Qe is more desirable.
Neuron Utilization U measures the percentage of neurons thatare the winner of one or more input vector in the map [3].
Thus, U nearer 1.0 is more desirable.

Fig. 5. Gray scale display of false-neighbor degrees and false-neighborweights. (c) False-neighbor degrees of FN-SOM. (d) False-neighbor weights
of WC-SOM.

We carry out the 30 simulations with different initial stateof the weight vectors and different order of inputting. The
averages of two measurements over 30 independent runs arelisted in Table I. It should be noted that the quantization
error Qe of WC-SOM is better than FN-SOM although theneuron utilization U of FN-SOM is the best value. This meansthat there are a few inactive neurons in the result of FNSOM, however, FN-SOM has not self-organized the statisticalfeatures of input data correctly. In other words, WC-SOM canobtain the clustering map reflecting the distribution state of
the input data more effectively than FN-SOM.
In order to evaluate how well SOM preserves the topologyof the data set, we calculate U-Matrix [5] which visualizes

the cluster structure of the map by showing distances between
the neighboring neurons. Figure 4 shows U-Matrices of fouralgorithms. We can see that the boundary lines of FN-SOM
and WC-SOM are clearer than other two algorithms, and it
is easy to distinguish between light areas (cluster) and darkarea (no input data) because there are few inactive neurons inthese results. However, it should be noted that it is clear that
U-Matrix of FN-SOM does not visualize the cluster structure
correctly. Meanwhile, U-Matrix of WC-SOM visualises thecluster structure effectively, although its boundary lines is veryclear. It means that FN-SOM can not preserve the topologyof the data set by increasing its FNDs forcibly, and WC-SOM
can obtain the clustering map reflecting the distribution state(a)- 65 -(b)

of the input data more effectively than FN-SOM.
This can be confirmed by the difference between FNDs
of FN-SOM and FNWs of WC-SOM after learning. Figure 5 shows FNDs and FNWs between the neurons displayedby gray-scale. From FNDs of FN-SOM as Fig. 5(a), we
can clearly see that even FNDs between the CORRECTneighboring neurons have been increased. This is becausethat all the FNDs between the neurons on same line are
increased simultaneously and forcibly, in FN-SOM algorithm.
On the other hand, in Fig. 5(b), FNWs of WC-SOM has beincreased flexibly depending on the shape of the input data incomparison with FN-SOM.
IV. C ONCLUSIONSThis study incorporates the weighted connections avoiding
false-neighbor effects into SOM (WC-SOM). WC-SOM has
false-neighbor weights (FNWs) allocated between connectionsof adjacent neurons to avoiding false-neighbor effect. In thealgorithm, the neighborhood distance has been proposed inaccordance with FNWs. We have compare WC-SOM withthe conventional SOM, Growing Grid and FN-SOM andhave confirmed that WC-SOM enables the most flexible selforganization by increasing FNWs with learning. Furthermore,
we have confirmed visually and numerically that WC-SOM
has few inactive neurons. In addition, FNWs of WC-SOM has
be increased flexibly depending on the shape of the input datain comparison with FN-SOM. These results mean that WCSOM can obtain the effective map reflecting the distribution
state of the input data using less neuron.
R EFERENCES[1] T. Kohonen, Self-organizing Maps, Berlin, Springer, 1995.
[2] B. Fritzke, “Growing Grid – a self-organizing network with constantneighborhood range and adaptation strength,” Neural Processing Letters,
vol. 2, no. 5, pp. 9–13, 1995.
[3] Y. Cheung and L. Law, “Rival-Model Penalized Self-Organizing Map,”
IEEE Trans. Neural Networks, vol. 18, no. 1, pp. 289–295, 2007.
[4] H. Matsushita and Y. Nishio, “Self-Organizing Map with False-Neighbor
Degree between Neurons for Effective Self-Organization,” IEICE Transactions on Fundamentals, vol. E91-A, no. 6, pp. 1463–1469, 2008.
[5] A. Ultsch, “Clustering with SOM: U*C,” Proc. of WSOM’05, pp. 75–82,
2005.

- 66 -