2007 International Symposium on Nonlinear Theory and itsApplications
NOLTA'07, Vancouver, Canada, September 16-19, 2007

Bifurcation of Nonlinear Spring Model of Self-Organizing Map
Haruna MATSUSHITA and Yoshifumi NISHIO
Department of Electrical and Electronic Engineering, Tokushima University
2-1 Minami-Josanjima, Tokushima 770-8506, JAPAN
Email: {haruna, nishio}@ee.tokushima-u.ac.jp
Abstract—The Self-Organizing Map (SOM) is an unsupervised neural network introduced by Kohonen and isa model simplifying self-organization process of the brain.
However, SOM is still far away from the realization of thebrain mechanism. In order to realize more powerful andmore flexible mechanism, it is important to propose newmodels of the brain mechanism and to investigate their behaviors. In our previous research, as the first step to realizea new nonlinear spring model of SOM, we have proposeda simple one dimensional 2-neuron model connected by anonlinear spring. In this study, in order to investigate thebehavior of the nonlinear spring model of SOM, we calculate one-parameter bifurcation diagram and the largest
Lyapunov exponent of the proposed model. Computer simulated results show that the neurons oscillate chaotically.

2. Self-Organizing Map (SOM)
We explain the learning algorithm of the Self-Organizing
Map (SOM). SOM consists of m neurons located at a regular low-dimensional grid, usually a 2-D grid. The basic SOM algorithm is iterative. Each neuron i has a ddimensional weight vector wi = (wi1 , wi2 , · · · , wid ) (i =1, 2, · · · , m). The initial values of all the weight vectors aregiven over the input space at random. The range of the elements of d-dimensional input data x j = (x j1 , x j2 , · · · , x jd )( j = 1, 2, · · · , N) are assumed to be from 0 to 1.
(SOM1) An input vector x j is inputted to all the neurons at
the same time in parallel.
(SOM2) Distances between x j and all the weight vectors
are calculated. The winner, denoted by c, is the neuron
with the weight vector closest to the input vector x j ;1. Introduction

c = arg min{kwi − x j k},

The Self-Organizing Map (SOM) is a subtype of artificial neural networks. It is trained using unsupervised learning to produce low dimensional representation of the training samples while preserving the topological properties of
the input space. SOM is introduced by Kohonen in 1982 [1]and is a model simplifying self-organization process of thebrain.
However, SOM is still far away from the realization of
the brain mechanism. In order to realize more powerful
and more flexible mechanism, it is important to propose
new models of the brain mechanism and to investigate theirbehaviors.
In our previous research, as the first step to realize a new
nonlinear spring model of SOM, we have proposed a simple one dimensional 2-neuron model connected by a nonlinear spring [2]. We have investigated its behavior undera simple assumption where input vectors are given to themodel periodically.
In this study, in order to investigate the behavior of
the nonlinear spring model of SOM, we calculate oneparameter bifurcation diagram and the largest Lyapunov
exponent of the proposed model. In Section 2, we explain
the learning algorithm of SOM. In Section 3, we explainproposed nonlinear spring model of SOM in detail. The
behaviors of the proposed model are investigated by calculating the projection of attractors and Poincaré map inSection 4. Furthermore, in order to investigate chaotic behavior in detail, one-parameter bifurcation diagram and thelargest Lyapunov exponent are calculated. Computer simulated results show that the neurons oscillate chaotically.

i(1)where k · k is the distance measure, Euclidean distance.
(SOM3) The weight vectors of the neurons are updated as;
wi (t + 1) = wi (t) + hc,i (t)(x j − wi (t)),(2)

where t is the learning step. hc,i (t) is called the neighborhood function and is described as a Gaussian function;)(kri − rc k2,(3)hc,i (t) = α(t) exp −2σ2 (t)where kri − rc k is the distance between map nodes c and i on
the map grid, α(t) is the learning rate, and σ(t) corresponds
to the width of the neighborhood function. Both α(t) andσ(t) decrease with time.
(SOM4) The steps from (SOM1) to (SOM3) are repeated
for all the input data.

- 525 -Winner c
Input vector

Winner c
Input vector

Figure 1: Learning process of Self-Organizing Map.

3. Nonlinear Spring Model of SOM [2]In our previous research, as the first step to realize a new
nonlinear spring model of SOM, we have proposed a simple one dimensional 2-neuron model connected by a nonlinear spring [2]. In this section, we explain the nonlinear
spring model in detail.
The model is shown in Fig. 2. The two neurons are assumed to have the same mass m and to be connected by anonlinear spring with the natural length l whose restoring
force F against the variation x̂ is represented by
F = −b x̂3

where t = 0 is the time when the input vector is given. The
shape of this function is shown in Fig. 4. Note that theother neuron does not receive a direct eﬀect from the inputvector.

fˆ (t )
Input

WinnerFigure 3: Input vector and winner.

(4)

where b denotes the stiﬀness of the spring.
Neuron 1

Neuron 2

12m v

m v

fˆ (t )

B̂

0lx̂Figure 2: Two-neuron nonlinear spring model of SOM.
Without loss of generality, we fix the position of the Neuron 1 as the origin of the x-coordinate. The position of theNeuron 2 ( x̂) and the velocities of the neurons (v1 and v2 )are chosen as the state variables. The motion equation ofthe model can be described as

d x̂

= v2

dt



 dv1(5)m= −av1 + b( x̂ − l)3

dt




dv


 m 2 = −av2 − b( x̂ − l)3dt
where a is the friction parameter. By changing the variablesand parameters;√√bbx̂ − l = x,v1 =y1 , v2 =y2 ,
mm
√
(6)
ma
t=
τ, k = √,bbm
the normalized equations are given as
 dx

= y2
dτ



 dy1
(7)
= −ky1 + x3


dτ



dy2


= −ky2 − x3 .
dτNext, we model the learning process of the SOM by theexternal force by input vectors. When an input vector isgiven to the 2-neuron model as Fig. 3, the winner, whichis the neuron nearer to the input vector, is attracted to theinput with the following force;√√()bmˆf (t) = B̂ sin(8)t0≤t<πmb0mπb 2tmπb

Figure 4: Force by external input vector.

In this study, in order to investigate the simplest learning process of the 2-neuron model, we concentrate on thecase that input vectors are given to the right-hand side andthe left-hand√ side of the model alternately with the fixed
frequency b/m/(2π). In this case, the motion equation ismodified as dx= y2dτ dy1(9)= −ky1 + x3 − f (τ)
dτ
dy2
= −ky2 − x3 + f (τ − π),dτ
wheref (τ) =B(sin τ + | sin τ|),2(10)andB̂
.
bThe shape of f (τ) is shown in Fig. 5.

- 526 -B=(11)f (τ )
B

0π

2π3π

τ

Figure 5: Force by periodic external input vector.

By using the Jacobian Matrix DT, we can calculate thelargest Lyapunov exponent;

4. Computer Simulation Results
In this section, we show some computer calculation results obtained by using Runge-Kutta method with time stepδt = 2π/500 for Eq. (9).
4.1. Attractors and Poincaré maps
The projections of attractors onto x–y2 plane are shownin Fig. 6(a). By changing parameter B, we can see that fourperiodic orbit (1), one-periodic orbit (3) and two-periodicorbit (4). Furthermore, we can confirm that the orbits of theattractors (2), (5) and (6) look chaos.
In order to investigate the chaotic behavior of the modelin detail, we define the Poincaré section as τ = 2nπ and plot
the discrete data on the Poincaré section onto x–y2 plane.
The Poincaré maps are shown in Fig. 6(b). Four-periodic
orbit (1) bifurcates to chaos which is estimated from theshape of the Poincaré map. Periodic window (3) and (4)can be observed in the chaos region. As B increases, thechaos (5) grows to more complex chaos (6). We can seethat the Poincaré map (5) and (6) are folded and have theshape like strange attractors [3].
4.2. Bifurcation Diagram and Lyapunov Exponent
In order to calculate the largest Lyapunov exponents ofthe attractors, we derive the variational equations of Eq. (9)as follows; d ∂xdτ ∂x0




d ∂y1



dτ
∂x0




d∂y2





dτ ∂x0




d ∂x




dτ ∂y1 0 d ∂y1dτ ∂y1 0





d ∂y2




dτ ∂y1 0




d ∂x


dτ
∂y2 0


d
∂y1


dτ∂y
20

d ∂y2




 dτ ∂y=
=====
=
=
=

20

∂y2
∂x0
∂y1
∂x−k+ 3x2
∂x0
∂x0
∂y2
∂x−k− 3x2
∂x0
∂x0
∂y2
∂y1 0
∂y1
∂x−k+ 3x2
∂y1 0
∂y1 0
∂y2
∂x−k− 3x2
∂y1 0
∂y1 0
∂y2
∂y2 0
∂y1
∂x−k+ 3x2
∂y2 0
∂y2 0
∂y2
∂x−k− 3x2.
∂y2 0
∂y2 0

(12)

∂x
∂y1 0
∂y1
∂y1 0
∂y2
∂y1 0

∂x 
∂y2 0 
∂y1 
.
∂y2 0 
∂y2 
∂y2 0N(14)where e( j) is a normalized basis.
One-parameter bifurcation diagram of x and the largest
Lyapunov exponents of the attractors, which are calculatedusing the algorithm of Shimada-Nagashima [5], are shownin Figs. 7(a) and (b), respectively. The control parameter
is B and another parameter k is fixed as 0.15. As shown inFig. 7(b), the Lyapunov exponent takes positive values fora wide range of B. Therefore, the nonlinear spring model
can be said to generate chaos, namely, the neurons oscillate chaotically. We observe the bifurcation in detail. For
6.3 < B < 6.92, a window corresponding to the periodic orbit in Fig. 6(1) is observed. For 6.92 < B < 7.4 , the largest
Lyapunov exponent takes positive values, namely, the attractor in Fig. 6(2) is chaos. For 8.9 < B < 10.5, the large
window corresponding to the period-doubling bifurcationin Figs. 6(3) and (4) is observed. For B < 10.5, chaotic
attractors can be observed for almost parameter values. Wehave also confirmed that a lot of small periodic windows
are embedded in the chaotic region.
5. Conclusions
In this study, we have investigated the behavior of thenonlinear spring model of SOM by calculating the projection of attractor and Poincaré map. Furthermore, in order
to investigate chaotic behavior in detail, one-parameter bifurcation diagram and the largest Lyapunov exponent werecalculated. We have investigated the related bifurcation
phenomena and have confirmed that the neurons oscillate
chaotically.
Our current works are changing the input method andincreasing the number of neurons. We investigate that how
the model behaves when the input vectors are given in a circular pattern. It is also our plan in future work to realize themechanism nearer to SOM, for example, like the nonlinear
spring model with a feature extraction capability.
References[1] T. Kohonen, Self-organizing Maps, Berlin, Springer, vol. 30,1995.

The Jacobian Matrix DT are obtained by integrating
Eq. (12) numerically [4]; ∂x ∂x0 ∂y1
DT =  ∂x0 ∂y2
∂x0

1 ∑log |DT( j) · e( j)|,N→∞ N
j=1

λ1 = lim(13)[2] H. Matsushita and Y. Nishio, “Nonlinear Spring Model ofSelf-Organizing Map and its Chaotic Behavior,” Proc. of ICCCAS, vol. 2, pp. 1099–1102, 2007.
[3] Y. Ueda, The Road to Chaos, Santa Cruz, Aerial Press, 1992.
[4] H. Kawakami, “Bifurcation of Periodic Responses in Forced
Dynamic Nonlinear Circuits: Computation of Bifurcation
Values of the System Parameters,” IEEE Trans. Circuits andSystems, vol. CAS–31, No. 3, pp. 248–260, 1984.
[5] I. Shimada and T. Nagashima, “A Numerical Approach to Ergodic Problem of Dissipative Dynamical Systems,” Progressof Theoretical Physics, vol. 61, No. 6, pp. 1605–1616, 1979.

- 527 -

1088664

42

20y2−2−4−4−6−6x1

2

3−10−34

864

42

200y2

10

6−2−2−4−4−6−6−8−8−2−10x12

3−10−34

1010

88664

42

2

00y2y2y208−2−2−4−4−6−6−8−8−10−3(4)−110−10−3(3)−8−2−2−10x12

3−10−3410108866442200y2y2−10−3−2−2−4−4−8−10−3y2(6)−2−10x1234101088664422y200x1234

32.52
1.5−2−10x1234

1−2−4−4−6−6−8−8−2−10x12

3−10−34101088664

42200−0.5−1−2−10x1

23−1.54−2
6−2−4−4−8−10−3−10−3−10x(a)123489B1011121310111213(a)0.5−2−10x12340−0.5−1
6−2−10x1234789B(b)Figure 7: Bifurcation diagram and Lyapunov exponent of
the nonlinear spring model. (a) One-parameter bifurcation
diagram. (b) Largest Lyapunov exponent. k = 0.15.

−6−8−27

10−2−60.5

0−2−10−3−1−6−8−10−3y2(5)y2−6−2x−8(2)0−2λ1y2(1)10−2−10x1234(b)Figure 6: Projection of attractors onto x–y2 plane and theirPoincaré maps. Fixed parameter k = 0.15. (a) Attractors.
(b) Poincaré maps. (1) B = 6.75. (2) B = 8. (3) B = 9.5.
(4) B = 10. (5) B = 10.4. (6) B = 12.

- 528 -