社団法人 電子情報通信学会
THE INSTITUTE OF ELECTRONICS,
INFORMATION AND COMMUNICATION ENGINEERS

信学技報
TECHNICAL REPORT OF IEICE.

Nonlinear Spring Model of Self-Organizing Map and its Chaotic Behavior
Haruna MATSUSHITA† and Yoshifumi NISHIO†† Department of Electrical and Electronic Engineering, Tokushima University
2–1 Minami-Josanjima, Tokushima 770–8506, Japan
E-mail: †{haruna,nishio}@ee.tokushima-u.ac.jp
Abstract The Self-Organizing Map (SOM) is an unsupervised neural network introduced by Kohonen and is amodel simplifying self-organization process of the brain. However, SOM is still far away from the realization of
the brain mechanism. In order to realize more powerful and more flexible mechanism, it is important to propose
new models of the brain mechanism and to investigate their behaviors. In this study, as the first step to realizea new nonlinear spring model of SOM, we propose a simple one dimensional 2-neuron model connected by a nonlinear spring. We investigate its behavior under a simple assumption where input vectors are given to the model
periodically. Furthermore, in order to investigate the behavior of the nonlinear spring model of SOM, we calculate
one-parameter bifurcation diagram and the largest Lyapunov exponent of the proposed model. Computer simulated
results show that the neurons oscillate chaotically.
Key words Self-Organizing Maps (SOM), Clustering, Data Mining

1. Introduction
The Self-Organizing Map (SOM) is a subtype of artificial
neural networks. It is trained using unsupervised learning toproduce low dimensional representation of the training samples while preserving the topological properties of the input
space. SOM is introduced by Kohonen in 1982 [1] and is amodel simplifying self-organization process of the brain.
However, SOM is still far away from the realization of thebrain mechanism. In order to realize more powerful and more
flexible mechanism, it is important to propose new modelsof the brain mechanism and to investigate their behaviors.
In this study, as the first step to realize a new nonlinear
spring model of SOM, we propose a simple one dimensional
2-neuron model connected by a nonlinear spring. We investigate its behavior under a simple assumption where input
vectors are given to the model periodically. Furthermore,in order to investigate the behavior of the nonlinear springmodel of SOM, we calculate one-parameter bifurcation diagram and the largest Lyapunov exponent of the proposedmodel. In Section II, we explain the learning algorithm ofSOM. In Section III, we explain proposed nonlinear springmodel of SOM in detail. The behaviors of the proposedmodel are investigated by calculating the projection of attractors and Poincaré map in Section IV. Furthermore, inorder to investigate chaotic behavior in detail, one-parameterbifurcation diagram and the largest Lyapunov exponent arecalculated. Computer simulated results show that the neurons oscillate chaotically.

Map (SOM). SOM consists of m neurons located at a regular low-dimensional grid, usually a 2-D grid. The basic SOM
algorithm is iterative. Each neuron i has a d-dimensional
weight vector wi = (wi1 , wi2 , · · · , wid ) (i = 1, 2, · · · , m).
The initial values of all the weight vectors are given overthe input space at random. The range of the elementsof d-dimensional input data xj = (xj1 , xj2 , · · · , xjd ) (j =1, 2, · · · , N ) are assumed to be from 0 to 1.
(SOM1) An input vector xj is inputted to all the neurons
at the same time in parallel.
(SOM2) Distances between xj and all the weight vectors
are calculated. The winner, denoted by c, is the neuron withthe weight vector closest to the input vector xj ;c = arg min{kwi − xj k},(1)iwhere k · k is the distance measure, Euclidean distance.
(SOM3) The weight vectors of the neurons are updated as;
wi (t + 1) = wi (t) + hc,i (t)(xj − wi (t)),(2)

where t is the learning step. hc,i (t) is called the neighborhood function and is described as a Gaussian function;

Winner cWinner c
Input vector

2. Self-Organizing Map (SOM)

Input vector

We explain the learning algorithm of the Self-Organizing

Fig. 1 Learning process of Self-Organizing Map.

—1—µhc,i (t) = α(t) exp −kr i − r c k2
2σ 2 (t)¶,(3)

where kr i −r c k is the distance between map nodes c and i on
the map grid, α(t) is the learning rate, and σ(t) corresponds
to the width of the neighborhood function. Both α(t) andσ(t) decrease with time.
(SOM4) The steps from (SOM1) to (SOM3) are repeated
for all the input data.

to the 2-neuron model as Fig. 3, the winner which is the neuron nearer to the input vector is attracted to the input withthe following force;rfˆ(t) = B̂ sin³btmq0<=t<mπb´(8)where t = 0 is the time when the input vector is given. Theshape of this function is shown in Fig. 4. Note that the otherneuron does not receive a direct effect from the input vector.

3. Nonlinear Spring Model of SOMfˆ (t )We consider a simple one dimensional 2-neuron model as
the first step to realize the nonlinear spring model of SOM.
The model is shown in Fig. 2. The two neurons are assumed
to have the same mass m and to be connected by a nonlinear spring with the natural length l whose restoring force F
against the variation x is represented by
F = −bx3InputWinnerFig. 3 Input vector and winner.

(4)fˆ (t )where b denotes the stiffness of the spring.
B̂Neuron 1Neuron 2m vm v0l12x̂0mb 2mbt

Fig. 2 Two-neuron nonlinear spring model of SOM.

Fig. 4 Force by external input vector.

Without loss of generality, we fix the position of the Neuron 1 as the origin of the x-coordinate. The position of theNeuron 2 (x̂) and the velocities of the neurons (v1 and v2 )are chosen as the state variables. The motion equation ofthe model can be described asIn this study, in order to investigate the simplest learning
process of the 2-neuron model, we concentrate on the casethat input vectors are given to the right-hand side and theleft-handpside of the model alternately with the fixed frequency
b/m/(2π). In this case, the motion equation ismodified as






dx̂
dt
dv1m

dt


 m dv2
dt=v2=−av1 + b(x̂ − l)3=3(5)−av2 − b(x̂ − l)

r
q

t=m
τ,
b

v1 =by1 ,mr
v2 =ak= √,bmby2 ,
mdy1

dτ
 dy2dτ=y2=−ky1 + x3=−ky2 − x3 .

dτ(6)

=y2

=−ky1 + x3 − f (τ )=−ky2 − x3 + f (τ − π),(9)wheref (τ ) =B(sin τ + | sin τ |),
2(10)

and
B=

the normalized equations are given asdx




 dτ

dy1

dτ


 dy2

where a is the friction parameter. By changing the variables
and parameters;x̂ − l = x,

dx


 dτ

B̂.
b(11)

The shape of f (τ ) is shown in Fig. 5.
(7)

Next, we model the learning process of the SOM by the external force by input vectors. When an input vector is given

4. Computer Simulation Results
In this section, we show some computer calculation results obtained by using Runge-Kutta method with time stepδt = 2π/500 for Eq. (9).

—2—

10

88

66

4

4y2y2

2

0−2−4−4−6−6−8−10−3

∂x0

0

12

3−10−34−2−10

10

8866

4

422y2y2(2)

0

∂y2
∂x0
∂y1
∂x−k+ 3x2∂x0
∂x0
∂x∂y2−k− 3x2
∂x0
∂x0
∂y2
∂y1 0
∂x
∂y1
−k+ 3x2
∂y1 0
∂y1 0
∂x
∂y2−k− 3x2
∂y1 0
∂y1 0
∂y2
∂y2 0
∂y1
∂x−k+ 3x2
∂y2 0
∂y2 0
∂x
∂y2
−k− 3x2.
∂y2 0
∂y2 0

−2−4−4−6−6−2

−10

12

3−10−34−2−10

x8

8664

4

2

2y2

100

−2−4−4==−8−10−3−2−10

12

3

4

=
=
=
=

101088664

4

∂x
∂y1 0
∂y1
∂y1 0
∂y2
∂y1 0

∂x
∂y2 0∂y1∂y2 0∂y2∂y2 0−2−10y2−4−4−6−6−10

123−10−34

34−2−10123412341234x10108866442200−2−2−4−4−6−6−8−8−10−3−10−3−2−10

1234−2−10xx

1010886644

2

200−2−2−4−4−6−6−8−8−2

By using the Jacobian Matrix DT, we can calculate the2−8−2x(13)10−2−10−3.
420−2−8(5)3x2(12)2−6−8−10−3y2=10−2−6(4)4x10y2=

3−8−10−3y2=

20−2−8(3)1x10−10−3∂x−1xThe Jacobian Matrix DT are obtained by integrating
Eq. (12) numerically [3]; ∂x0 ∂y1DT =  ∂x0 ∂y2−2x(6)0−2−8y2dτ ∂y1 0d ∂y2dτ
∂y1 0
d∂x
dτ∂y20
d ∂y1dτ ∂y2 0 d ∂y2dτ ∂y2 0(1)y2

 d ∂xdτ ∂x0d ∂y1

dτ
∂x0


d∂y2



dτ∂x
0


d∂x




dτ ∂y1 0 d ∂y1

102y2

4. 1 Attractors and Poincaré maps
The projections of attractors onto x–y2 plane are shown in
Fig. 6(a). By changing parameter B, we can see that fourperiodic orbit (1), one-periodic orbit (3) and two-periodicorbit (4). Furthermore, we can confirm that the orbits of theattractors (2), (5) and (6) look chaos.
In order to investigate the chaotic behavior of the modelin detail, we define the Poincaré section as τ = 2nπ and plot
the discrete data on the Poincaré section onto x–y2 plane.
The Poincaré maps are shown in Fig. 6(b). Four-periodicorbit (1) bifurcates to chaos which is estimated from theshape of the Poincaré map. Periodic window (3) and (4) canbe observed in the chaos region. As B increases, the chaos(5) grows to more complex chaos (6). We can see that thePoincaré map (5) and (6) are folded and have the shape likestrange attractors [2].
4. 2 Bifurcation Diagram and Lyapunov Exponent
In order to calculate the largest Lyapunov exponents of
the attractors, we derive the variational equations of Eq. (9)as follows;−101234−10−3−2−10xx(a)(b)Fig. 6 Projection of attractors onto x–y2 plane and their Poincarémaps. Fixed parameter k = 0.15. (a) Attractors.
(b) Poincaré maps. (1) B = 6.75. (2) B = 8. (3) B = 9.5.
(4) B = 10. (5) B = 10.4. (6) B = 12.

largest Lyapunov exponent;
1 X
log |DT(j) · e(j)|,N →∞ NNλ1 = limf( )
B

0(14)

j=1



2


3


Fig. 5 Force by periodic external input vector.

where e(j) is a normalized basis.
One-parameter bifurcation diagram of x and the largest
Lyapunov exponents of the attractors, which are calculatedusing the algorithm of Shimada-Nagashima [4], are shown in—3—

Figs. 7(a) and (b), respectively. The control parameter is
B and another parameter k is fixed as 0.15. As shown inFig. 7(b), the Lyapunov exponent takes positive values fora wide range of B. Therefore, the nonlinear spring model
can be said to generate chaos, namely, the neurons oscillate chaotically. We observe the bifurcation in detail. For
6.3 < B < 6.92, a window corresponding to the periodic orbit in Fig. 6(1) is observed. For 6.92 < B < 7.4 , the largest
Lyapunov exponent takes positive values, namely, the attractor in Fig. 6(2) is chaos. For 8.9 < B < 10.5, the large
window corresponding to the period-doubling bifurcation inFigs. 6(3) and (4) is observed. For B < 10.5, chaotic attractors can be observed for almost parameter values. We
have also confirmed that a lot of small periodic windows areembedded in the chaotic region.

attractor and Poincaré map. In order to investigate chaotic
behavior in detail, one-parameter bifurcation diagram andthe largest Lyapunov exponent were calculated. We have investigated the related bifurcation phenomena and have confirmed that the neurons oscillate chaotically.
References[1] T. Kohonen, Self-organizing Maps, Berlin, Springer, vol. 30,1995.
[2] Y. Ueda, The Road to Chaos, Santa Cruz, Aerial Press,
1992.
[3] H. Kawakami, “Bifurcation of Periodic Responses in Forced
Dynamic Nonlinear Circuits: Computation of Bifurcation
Values of the System Parameters,” IEEE Trans. Circuitsand Systems, vol. CAS–31, No. 3, pp. 248–260, 1984.
[4] I. Shimada and T. Nagashima, “A Numerical Approachto Ergodic Problem of Dissipative Dynamical Systems,”
Progress of Theoretical Physics, vol. 61, No. 6, pp. 1605–1616, 1979.

32.5
21.5

x

10.5
0−0.5
−1−1.5−26

7891011121310111213B

(a)
10.5

0−0.5

−1
6

7

89B

(b)
Fig. 7 Bifurcation diagram and Lyapunov exponent of the nonlinear spring model. (a) One-parameter bifurcation diagram.
(b) Largest Lyapunov exponent. k = 0.15.

5. Conclusions
In this study, as the first step to realize a new nonlinear
spring model of SOM, we have proposed a simple one dimensional 2-neuron model connected by a nonlinear spring.
We have investigated its behavior under a simple assumption where input vectors are given to the model periodically.
Furthermore, we have investigate the behavior of the nonlinear spring model of SOM by calculating the projection of—4—