Self-Organizing Map
Considering False Neighboring Neuron
Haruna MATSUSHITA∗ and Yoshifumi NISHIO∗∗ Department of Electrical and Electronic Engineering, Tokushima University

2–1 Minami-Josanjima, Tokushima 770–8506, Japan
Telephone: +81–88–656–7470, Fax: +81–88–656–7471,
Email: {haruna, nishio}@ee.tokushima-u.ac.jp
Abstract— In the real world, it is not always true that the nextdoor house is close to my house, in other words, “neighbors”
are not always “true neighbors”. In this study, we propose a
new Self-Organizing Map (SOM) algorithm which considers theFalse Neighboring Neuron (called FNN-SOM). The FNN-SOM
self-organizes with considering the real neighboring relation. The
behavior of FNN-SOM is investigated with learning for various
input data. We confirm that we can obtain the more effective
map reflecting the distribution state of input data than theconventional SOM.

I. I NTRODUCTION
Since we can accumulate a huge amount of data in recent
years, it is important to investigate various clustering methods.
Then, the Self-Organizing Map (SOM) has attracted attention
for its clustering properties. SOM is an unsupervised neuralnetwork introduced by Kohonen in 1982 [1] and is a model
simplifying self-organization process of the brain. SOM obtains statistical feature of input data and is applied to a wide
field of data classifications. We can obtain the map reflecting
the distribution state of input data using SOM. In the learningalgorithm of SOM, a winner, which is a neuron with theweight vector closest to the input vector, and its neighboring
neuron are updated, regardless of the distance between the
input vector and the neighboring neuron. For this reason, if we
apply SOM to clustering of the input data which includes someclusters located at distant location, there are some inactive
neurons between clusters. Because inactive neurons are on apart without the input data, we are misled into thinking that
there are some input data between clusters.
Meanwhile, in the real world, it is not always true that the
next-door house is close to my house. For example, the casethat the next-door house is at the top of a mountain whereas
my house is at the foot (as Fig. 1(a)), and other case that there
is a river, which does not have a bridge, between my house andmy next-door house (as Fig. 1(b)). This means, “neighbors”
are not always “true neighbors”.
In addition, the synaptic strength is not constant in the brain.
There is scarcely any research changing the synaptic strength.
In this study, we propose a new SOM algorithm which
considers the False Neighboring Neuron (called FNN-SOM).
A false-neighbor degree is allocated between each neuron of
FNN-SOM. The neuron, which is the most distant from the
input data in a set of direct topological neighbors of winner,
1-4244-0921-7/07 $25.00 © 2007 IEEE.

B

A
(a)

C

B

A

C

(b)

Fig. 1. What is the “neighbors”? The house B and C is A’s next-door
neighbor on the left and on the right, respectively. (a) The house B is at the
top of a mountain. (b) The river between A and B does not have a bridge.

is said to be “false neighboring neuron (FNN)”. The falseneighbor degrees of FNN and its neighbors are increased,
and the false-neighbor degrees act as a burden of the distance
between each map node when the weight vectors of neurons
are updated. Therefore, the FNN-SOM self-organizes withconsidering the real neighboring relation.
In Section II, we explain the learning algorithm of theconventional SOM. In Section III, the learning algorithm of
the proposed SOM, FNN-SOM, is explained in detail. The
learning behaviors of FNN-SOM for various input data areinvestigated in Section IV. We can see that there are no inactive
neurons using FNN-SOM, and FNN-SOM can obtain the more
effective map reflecting the distribution state of input data thanthe conventional SOM. In Section V, FNN-SOM are applied
to clustering. Furthermore, clustering ability is evaluated byvisualization of results for the iris data.
II. S ELF -O RGANIZING M AP (SOM)
We explain the learning algorithm of the conventional
SOM. SOM consists of m neurons located at a regular lowdimensional grid, usually a 2-D grid. The basic SOM algorithm is iterative. Each neuron i has a d-dimensional weightvector wi = (wi1 , wi2 , · · · , wid ) (i = 1, 2, · · · , m). The initialvalues of all the weight vectors are given over the input spaceat random. The range of the elements of d-dimensional input
data xj = (xj1 , xj2 , · · · , xjd ) (j = 1, 2, · · · , N ) are assumed
to be from 0 to 1.
(SOM1) An input vector xj is inputted to all the neurons at
the same time in parallel.
(SOM2) Distances between xj and all the weight vectors are

1533

2

calculated. The winner, denoted by c, is the neuron with theweight vector closest to the input vector xj ;c = arg min{wi − xj },
i

1

(1)

0

where  ·  is the distance measure, Euclidean distance.
(SOM3) The weight vectors of the neurons are updated as;
wi (t + 1) = wi (t) + hc,i (t)(xj − wi (t)),

where r i − r c  is the distance between map nodes c and i on
the map grid, α(t) is the learning rate, and σ(t) corresponds
to the width of the neighborhood function. Both α(t) and σ(t)decrease with time as follows;


t/T
t/T
σ(T )
α(T )
, σ(t) = σ(0), (4)α(t) = α(0)
α(0)
σ(0)
where T is the maximum number of the learning.
(SOM4) The steps from (SOM1) to (SOM3) are repeated for
all the input data.

(a)

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

(b)

neighbors N c1 without l are set to zero (this means “refresh”
the connection reference).
Cf (c,i) = 0,(FNN-SOM1) An input vector xj is inputted to all the neurons
at the same time in parallel.
(FNN-SOM2) Distances between xj and all the weight vectors are calculated, and the rank order of distances, denoted by
k = 0, · · · , m − 1, is calculated. ki is the rank of wi , namely,kc = 0 is the rank of the winner c which is closest to xj
according to Eq. (1).
(FNN-SOM3) A False Neighboring Neuron l (called FNN) is
found. FNN is the most distant from xj in N c1 . However, if
kl is smaller than the number of N c1 , the FNN l dose notexist.
i ∈ N c1 ,(5)
l = arg max{wi − xj },iwhere N c1 is the set of direct topological neighbors of c as
Fig. 2(a).
The false-neighbor degree between c and l is increased;(6)

Furthermore, the false-neighbor degrees between c and the setof neurons Sl , which are located beyond l as Fig. 2(b), arealso increased;(7)(FNN-SOM4) The false-neighbor degrees between c and its

i ∈ N c1 ∩ i = l.

(8)Furthermore, the false-neighbor degrees between c and the setof neurons Sc , which are located beyond the neurons in N c1
except l as Fig. 2(b), are also set to zero;Cf (c,i) = 0,

We explain a proposed new SOM algorithm, FNN-SOM. A
false-neighbor degree Cf (c,i) is allocated between each neuronof FNN-SOM. The initial values of all of the false-neighbor
degree are set to zero, and the initial values of all the weightvectors are given over the input space at random.

i ∈ Sl .

3

Fig. 2.
Neighborhood distances of the rectangular grid. (a) Discreteneighborhoods N c0 , N c1 and N c2 of the centermost neuron. (b) Neighboringreferences of c = 18. N c1 = [13, 17, 19, 23]. If l = 13, Sl = [3, 8] andSc = [16, 20]. If l = 17, Sl = 16 and Sc = [3, 8, 20]. If l = 19, Sl = 20
and Sc = [3, 8, 16]. If l = 23, Sl does not exist and Sc = [3, 8, 16, 20].

III. SOM CONSIDERING FALSE N EIGHBORING N EURON(FNN-SOM)

Cf (c,i) = Cf (c,i) + 1,

2

(2)

where t is the learning step. hc,i (t) is called the neighborhood
function and is described as a Gaussian function;r i − r c 2
hc,i (t) = α(t) exp −,(3)
2σ 2 (t)Cf (c,l) = Cf (c,l) + 1.

1i ∈ Sc .

(9)(FNN-SOM5) The weight vectors of the neurons are updatedas;(10)
wi (t + 1) = wi (t) + hnc,i (t)(xj − wi (t)),
where hnc,i (t) is the neighborhood function of the proposed
SOM;


λc,i
hnc,i (t) = α(t) exp − 2,(11)4σ (t)
where

λc,i = 0.5ki + (r i − r c 2 + Cf (c,i) ).

(12)(FNN-SOM6) The steps from (FNN-SOM1) to (FNN-SOM5)
are repeated for all the input data.
IV. A PPLICATION TO F EATURE E XTRACTION
A. For Lorenz data
We carry out learning simulation for the data generated
by Lorenz equations which is three dimensional nonlinear
differential equations and is a chaotic system described by
Edward Lorenz [2].
In this study, we use 1st and 2nd dimensional data of Lorenz
attractor shown in Fig. 3(a), and is like a butterfly. The total
number of data N is 1000. All the input data are sorted byrandom.
The learning conditions are as follows. Both the conventional SOM and FNN-SOM has m = 289 (17×17). We repeat
the learning 15 times for all input data, namely T = 15000.
The parameters of the learning are chosen as follows;(For SOM)

1534

1

1

1

0.9

0.9

0.9

0.8

0.8

0.8

0.7

0.7

0.7

0.6

0.6

0.6

0.5

0.5

0.5

0.4

0.4

0.4

0.3

0.3

0.3

0.2

0.2

0.2

0.1

0.1

0
0

0.2

0.4

0.6

0.8

0.1

0
0

1

0.2

(a)

0.4

0.6

0.8

0
0

1

0.2

0.4

(b)

0.6

0.8

1

(c)

Fig. 3. Learning for 2-D data generated by Lorenz equations. (a) Input data. (b) Simulation result of conventional SOM. (c) Simulation result of FNN-SOM.

0.6

0.6

0.6
Z

1
0.8

Z

1
0.8

Z

1
0.8

0.4

0.4

0.4

0.2

0.2

0.2

0
1

0
1

0
1

1
0.5

0.5

0.5

Y
0 0

1

X

(a)
Fig. 4.

X

0 0

(b)

1
0.5

0.5

Y

0.5

Y
0 0

X

(c)

Learning for 3-D data generated by Langford equations. (b) Simulation result of conventional SOM. (c) Simulation result of FNN-SOM.

α(0) = 0.5, σ(0) = 5, α(T ) = σ(T ) = 0(For FNN-SOM)
α(0) = 0.5, α(T ) = 0.05, σ(0) = 10, σ(T ) = 0.01.
The results of SOM and FNN-SOM are shown in Figs. 3(b)and (c). The conventional SOM can not self-organizes the edge
data of the input space because the center are denser area thanthe edge. However, FNN-SOM self-organizes the center data
as well as the edge data of the input space because the neuron
located at distant location from winner is defined FNN.
B. For Langford data
Next, we carry out learning simulation for the data generatedby Langford equation [3]. This attractor is 2-torus and has thecavity. Figure 4(a) shows the input data. The total number ofdata N is 1000.
The learning results of SOM and FNN-SOM are shown in
Figs. 4(b) and (c). We can see that FNN-SOM can obtain the
more effective map reflecting the distribution state of input
data than SOM.
V. A PPLICATION TO C LUSTERING
We apply FNN-SOM to clustering.
A. For 2-dimensional input data
First, we consider 2-dimensional input data generated by
the probability density function as shown in Fig. 5(a). Thisdata has 5 clusters, however, it is difficult to find 5 clusters
because the each cluster is close to each other and the densityof each clusters is different. Total number of the input data N
is 700. The learning conditions are the same used in Fig. 3.
The simulation results of the conventional SOM and FNNSOM are shown in Figs. 5(b) and (c). We can see that there
are some inactive neurons between two clusters in the resultof SOM, however, there are few inactive neurons in the resultof FNN-SOM. This is because the neurons of FNN-SOM are
not affected by FNN, so, the neurons can learn more distantfor the distant input data, than SOM learning.
Figure 6 shows distances between neighboring neurons andthus visualizes the cluster structure of the map. Black circles
on this figure mean large distance between neighboring map
nodes. Clusters are typically uniform areas of white circles.
Each map is normalized by each distance, respectively.
We can see that the boundary line of FNN-SOM is clearer
than the conventional SOM because FNN-SOM has few inactive neurons between clusters. Therefore, we can confirm thatthe input data has 5 clusters from Fig. 6(b).
B. For Iris data
Furthermore, we apply FNN-SOM to the real world clustering problem. We use the Iris plant data [4] as real data.
This data is one of the best known databased to be foundin pattern recognition literatures [5]. The data set contains
three clusters of 50 instances respectively, where each classrefers to a type of iris plant. The number of attributes isfour as the sepal length, the sepal width, the petal length andthe petal width, namely, the input data are 4-dimension. The
three classes correspond to Iris setosa, Iris versicolor and Iris

1535

1

1

1

0.9

0.9

0.9

0.8

0.8

0.8

0.7

0.7

0.7

0.6

0.6

0.6

0.5

0.5

0.5

0.4

0.4

0.4

0.3

0.3

0.3

0.2

0.2

0.2

0.1

0.1

0
0

0.2

0.4

0.6

0.8

0
0

1

0.1
0.2

(a)

0.4

0.6

(b)

0.8

1

0
0

0.2

0.4

0.6

0.8

1

(c)

Fig. 5.

Clustering for 2-dimensional input data. (a) Input data. (b) Simulation result of the conventional SOM. (c) Simulation result of FNN-SOM.

Fig. 6.

Visualization of result. (a) Conventional SOM. (b) FNN-SOM.

(a)

(b)

virginica, respectively. Iris setosa is linearly separable fromthe other two, however Iris versicolor and Iris virginica are
not linearly separable from each other.
The learning conditions are as follows. Both the conventional SOM and FNN-SOM has m = 100 (10×10). We repeat
the learning 100 times for all input data, namely T = 15000.
The parameters of the learning are chosen as follows;(For SOM)(a)α(0) = 0.5, σ(0) = 3, α(T ) = σ(T ) = 0(For FNN-SOM)
α(0) = 0.5, α(T ) = 0.05, σ(0) = 10, σ(T ) = 0.01.
The visualizations of results are shown in Fig. 7. The
boundary line of FNN-SOM is clearer than the conventional
SOM, and from FNN-SOM result, we can confirm that 3
clusters exists.
VI. C ONCLUSIONSIn this study, we have proposed a new SOM algorithm
which considers the False Neighboring Neuron (called FNNSOM). The neuron, which is the most distant from the input
data in a set of direct topological neighbors of winner, isdefined as “false neighboring neuron (FNN)”. Furthermore,
the false-neighbor degree is allocated between each neuronof FNN-SOM. The false-neighbor degrees of FNN and its
neighbors are increased, and the false-neighbor degrees act
as a burden of the distance between each map node when theweight vectors of neurons are updated. We have investigated its
behaviors with learning for various input data and application
to clustering. We have confirmed the efficiency of FNN-SOM.

(b)Fig. 7. Visualization of result for Iris data. Label A, B and C correspondto Iris setosa, Iris versicolor and Iris virginica, respectively. (a) Conventional
SOM. (b) FNN-SOM.

R EFERENCES[1] T. Kohonen, Self-organizing Maps, Berlin, Springer, vol. 30, 1995.
[2] E. N. Lorenz, “Deterministic Nonperiodic Flow”, Journal of the Atmospheric Sciences, vol.20, pp. 130-141, 1963.
[3] W. F. Langford, “Numerical Studies of torus bifurcations,” International
Series of Numerical Mathematics, vol.70, pp.285-295, 1984.
[4] UCI Repository of Machine Learning Databases and Domain Theories,
FTP address: ftp://ftp.ics.uci.edu/pub/machine-learning-databases[5] R. A. Fisher, “The Use of Multiple Measurements in Taxonomic Problems,” Annual Eugenics, no.7, part II, pp. 179-188,1936.

1536