2005 RISP International Workshop on Nonlinear
Circuit and Signal Processing (NCSP'05)
Hawaii, USA, Mar. 4-6, 2005.

Research on Improvement in Self-Organization Capability Using Two SOMs
Haruna MATSUSHITAy , Yoko UWATEy and Yoshifumi NISHIOyy

Department of Electrical and Electronics Engineering
Tokushima University
2-1 Minami-Josanjima, Tokushima, Japan
Phone:+81-88-656-7470, Fax:+81-88-656-7471
Email: fharuna, uwate, nishiog@ee.tokushima-u.ac.jp

weight vector wi = (wi1; wi2; 1 1 1 ; wil) (i = 1; 2; 1 1 1 ; m)of the neuron i, according to:The Self-Organizing Map (SOM) is unsupervised neural networks introduced by Kohonen in 1982 [1]. Selfnetji = kw i 0 xj k01 :(1)organization is to change an internal structure to adjust
to the signal from the outside though teacher signals are In this study, Euclidean distance is used for (1).
not given from the outside. SOM is a model simplifying (SOM4) The neuron with the maximum internal activself-organization process of the brain. SOM obtains a ity degree is winner neuron c. In other words, the winner
statistical feature of input data and is applied to a wide neuron c is the neuron with the weight vector nearest to
eld of data classi cations. And, the e ectiveness is said the input vector.
(SOM5) The weight vector of the winner neuron c andto be very high [2]-[7].
theneighborhood neuron Nc (as Fig. 1(b)) are updated
SOM has the position relation between neurons withas:d-dimensions, and the neuron which won a competition,and some neurons around the winner learn. For exam- wi (t + 1) = wi (t) + (t)(xj 0 wi (t)); i 2 Nc (t); (2)ple, if input data are 2-dimensional random data which
are uniformly-distributed between 0 and 1, SOM is dis- where (t) is the learning coecient. (t) and Nc (t) aretributed like a square. Thus, a distribution of input data decreased with time according to the following equations:can be expressed with a weight vector by learning. However, when input data is nonuniform, some experience is(t) = 0 (1 0 t=T ) ;(3)needed to extract the dense part.
Nc (t) = [Nc (0) (1 0 t=T )] ;(4)
In this research, in order to solve this problem, wepropose a method of using simultaneously two kinds of where [ ] denotes the Gauss' notation, T is the maximum
SOMs whose features are di erent. One is distributed number of learning, 0 is the initial value of the learningin the area for which input data gathers and the other coecient, Nc (0) is the initial value of the neighborhood
over the whole. As a result, the feature of data can be coecient.
extracted more e ectively by using two kinds of SOMs. (SOM6) The steps from (SOM2) to (SOM5) are repeated for all the input vectors.
1. Introduction

2. Self-Organizing Map

3. Two SOMs

We concretely explain the learning method of SOM. mneurons are arranged on a 2-dimensional grid as shown In this study, we propose a method of extracting thein Fig. 1(a). The range of the elements of l-dimensional feature of the input data more e ectively using two kinds
input vectors xj = (xj 1; xj 2; 1 1 1 ; xjl)(j = 1; 2; 1 1 1 ; N ) of SOMs whose features are di erent. Namely, one selforganizes the area on which input data are concentrated,
are assumed to be from 0 to 1.
(SOM1) The initial values of all the weight vectors w and the other self-organizes the whole of the input space.
We call the former SOML and the latter SOMG .
are given between 0 and 1 at random.
We explain the learning method of the proposed(SOM2) An input vector xj is inputted to all the neuSOMs.
rons at the same time in parallel.
(SOM3) The internal activity degree netji is calculated (2-SOM1) The initial values of all the weight vectors
from the distance between the input vector xj and the wL of SOML are given between 0 and 1 at random. The- 307 -(a) Structure of SOM.

(b) Example of neighborhood area.

Figure 1: Self-organiging map.

1
0.8
0.6
0.4
0.2
00

0.2

0.4

0.6

0.8

1Figure 2: Example of initial states of two SOMs

are updated as:
wLi (t + 1) = wLi (t) + L (t)(xj 0 wLi(t)); i 2 NLc (t);(8)
where L (t) is the learning coecient. G (t), L (t),NG c , and NL c are decided as:(9)
G (t) = G 0 (1 0 t=T ) ;n1o2 ;(10)
L (t) = L 0 1 0 (t=T )hNGc (t) = NGc (0)

N



i

Gc (0)01 (1 0 t=T ) ;NGc (0)0n
NGc (t) < NGc (0);(11)

NLc (t) = [NLc(0) (1 0 t=T )] ;(12)
initial values of all the weight vectors wG of SOMG aregiven between 0.4 and 0.6 at random, as Fig. 2.
where G 0 and L 0 are the initial values of learning(2-SOM2) An input vector xj is inputted to all the coecients, NGc (0) and NLc (0) are the initial values
neurons of SOMG and SOML at the same time in paral- of neighborhood coecients and n is the number of allSOMs.
lel.
(2-SOM3) The distance between xj and the weight (2-SOM6) The steps from (2-SOM2) to (2-SOM5) arevector wG = (wGi1 ; wGi2; 1 1 1 ; wGil) of the neuron i of repeated for all the input vectors.
SOMG, and the distance between xj and the weight vec- (2-SOM7) Furthermore, only SOM
G learns. Time isjjtor wL = (wLi1; wLi2; 1 1 1 ; wLil ) of the neuron i of SOM
L reset as t = 0. We compute netG i and netL i according
jare calculated. The internal activity degree netG i and to the steps (2-SOM2) and (2-SOM3).
(2-SOM8) If the distance between xj and the neuron
netL ji are obtained as:of SOML with the maximum netL is larger than a small
netG ji = kw Gi 0 xj k01 ;(5) distance ", the weight vector of the neuron of SOMG
with the maximum netG and the neighborhood neuron
NGsc are updated as:netL ji = kw Li 0 xj k01 :(6)wGi(t + 1) = wGi (t) + Gs (t)(xj 0 wGi(t)); (13)i 2 NGsc(t); max(netLji )  1" ;(2-SOM4) The winner neuron c is the neuron with themaximum internal activity degree in all netG and netL . where Gs (t) is the learning coecient.
(2-SOM5) If the winner neuron c is the neuron in (2-SOM9) The steps from (2-SOM7) to (2-SOM8) are
SOMG, the weight vector of the winner neuron and the repeated for all the input vectors. Gs (t) and NGsc (t)neighborhood neuron NGc are updated as:
are decreased with time according to the following equations:wGi (t + 1) = wGi (t) + G (t)(xj 0 wGi (t)); i 2 NGc (t);(14)
Gs (t) = Gs 0 (1 0 t=Ts ) ;(7)
NGsc (t) = [NGsc (0) (1 0 t=Ts )] ;(15)
where G (t) is the learning coecient.
If the winner neuron c is in SOML , the weight vector where Ts is the maximum number of learning since stepof the winner neuron and the neighborhood neuron NL c (2-SOM7),- 308 -4. Simulation Results
4.1. Input Data

Input data is 2-dimensional random data of 450 points
whose distribution is non-uniform as Fig. 3. 225 points
are distributed within a small range from 0.2 to 0.45 horizontally and from 0.7 to 0.95 vertically. The remaining
225 points are uniformly distributed between 0 and 1.

1

1

0.8

0.8

0.6

0.6

0.4

0.4

0.2

0.2

00

0.2

0.4

0.6

0.8

(a) One SOM.

1

00

0.2

0.4

0.6

0.8

1(b) Two SOMs.

Figure 4: Simulation Results.

10.80.6

5. n SOMs

0.4

The number of SOML can be increased depending on
distribution of input data. We carry out simulation forvarious input data. For instance, when the data of Fig.
Figure 3: 2-dimensional input data.
5(a) are input, one SOMG and two SOML are e ective.
The learning method for 2 or more SOML is similar tothe case of Two SOMs.
4.2. Setting
Figure 6 shows the respective results. In all the results,
SOML is self-organized in the area for which input data
4.2.1. One SOM
concentrate, and SOMG is self-organized over the wholespace.
SOM has 441 neurons (21 2 21). The initial values of input
Weconsider
that the concept using two kinds of SOMs
the learning coecient and the neighborhood coecient can be used to extract
the data only in a dense part of theare chosen as follows:
input data, because the SOML can nd such an area bythemselves. Figures 7(a) and (b) show one of the SOML
0 = 0:9; Nc (0) = 7:
the data whose nearest neuron is in the SOML, reIn the learning process, all the input data are given 8 andspectively,after the self-organization in the previous simtimes repeatedly.
ulation of Fig. 6(c). The data in Fig. 7(b) is almost sameas one of the dense parts of the input data in Fig. 5(c).
4.2.2. Two SOMs
We can also extract other dense parts of the input data
SOMG and SOML have 225 neurons (15 2 15) each, in Fig. 5(c) using the other SOML as shown in Fig. 8.
namely, 2 SOMs have totally 445 neurons. The initialvalues of the learning coecients, the neighborhood co- 6. Conclusions
ecients and the small distance are chosen as follows:
In this study, we proposed the method of extracting
G 0 = L 0 = 0:9; NGc (0) = NLc (0) = 6;
the feature of the input data more e ectively using thetwo kinds of SOMs whose features are di erent. TheGs 0 = 0:7; NGsc (0) = 4; " = 0:02:
numberof SOML is increased depending distribution ofAfter we give all the input data to SOMG and SOML
input
data.
We con rmed by computer simulations that
4 times, furthermore we give them for learning only ofnSOMs
couldextract the dence parts of the input data.
SOMG 4 times repeatedly.
In the future, we hope to develop this method further
and to apply it to various data.
4.3. Results
The simulation result is shown in Fig. 4. From Fig.
4(a), we can say that One SOM is self-organized. How- References
ever, Two SOMs can be self-organized more e ectively as [1] T. Kohonen, \Self-Organising Maps," 2nd ed, Springer
Series in Information Sciences, Vol. 30, Springer, Berlin,
shown in Fig. 4(b). In addition, SOML is self-organizedin the area for which input data concentrate, and SOMG [2] 1995.
B. Angeniol, G. de La C. Vaubois and J. Y. Le Texier,is self-organized over the whole input space.
\Self-Organizing Feature Maps and the Travelling Sales0.2

00

0.2

0.4

0.6

0.8

1- 309 -

1

1

1

0.8

0.8

0.8

0.6

0.6

0.6

0.4

0.4

0.4

0.2

0.2

0.2

00

0.2

0.4

0.6

0.8

1

00

0.2

0.4

0.6

0.8

100

0.2

0.4

0.6

0.8

1(a) With 2 dense parts. (b) With 3 dense parts. (c) With 4 dense parts.

Figure 5: Input data with various distributions.
111

0.8

0.8

0.8

0.6

0.6

0.6

0.4

0.4

0.4

0.2

0.2

0.2

000.2

0.4

0.6

0.8

1000.2(a) 3 SOMs.

0.4

0.6

0.8

1

000.2

(b) 4 SOMs.

0.4

0.6

0.8

1(c) 5 SOMs.

Figure 6: Self-organization by n SOMs.

11

0.8

0.8

11

0.6

0.6

0.8

0.8

0.4

0.4

0.6

0.6

0.2

0.2

0.4

0.4

0.2

0.2

000.2

0.4

0.6

0.8

(a) SOML1 .

1000.20.4

0.6

0.8

1(b) Extracted data.

00Figure 7: Extraction of dense parts of input data by SOML .

0.2

0.4

0.6

0.8

1000.2

0.4

0.6

0.8

1(a) Extraction by SOML2 . (b) Extraction by SOML3 .
10.8

man Problem," Neural Networks, Vol. 1, pp. 289-293,
1988[3] H. Tokutaka, \Condensed Review of SOM and LVQ Research in Japan," Proc. of WSOM'97, pp. 322-329, 1997.
[4] M. Cottrell, E. de Bodt, M. Verleysen, \Kohonen Maps
Versus Vector Quantization for Data Analysis," Proc. ofESANN'97, pp. 187-193, 1997.
[5] G. Simon, A. Lendasse, M. Cottrell, J.-C. Fort, M. Verleysen, \Double SOM for Long-term Time Series Prediction," Proc. of WSOM'03, pp. 35-40, 2003.
[6] M. Strickert and B. Hammer, \Neural Gas for Sequences," Proc. of WSOM'03, pp. 53-58, 2003.
[7] T. Ehara, H. Sasamura, T. Saito, \An Approach to theTSP based on Growing Self-Organizing Maps," Proc. ofNOLTA'04, pp. 709-712, 2004.

- 310 -0.6
0.4
0.2
000.2

0.4

0.6

0.8

1(c) Extracted by SOML 4.

Figure 8: Dense parts extraction.