2006 International Joint Conference on Neural Networks
Sheraton Vancouver Wall Centre Hotel, Vancouver, BC, Canada
July 16-21, 2006

Behavior of Fatigable SOM and its Application to Clustering
Masato Tomita, Non-member, IEEE, Haruna Matsushita, Student member, IEEE ,and Yoshifumi Nishio, Member, IEEE
Abstract— The Self-Organizing Map (SOM) is popular algorithm for unsupervised learning and visualization introduced
by Teuvo Kohonen. One of the most attractive applications of
SOM is clustering and several algorithms for various kinds ofclustering problems have been reported and investigated. In thisstudy, we propose a new type of SOM algorithm, which is called
Fatigable SOM (FSOM) algorithm. The important feature of
FSOM is that the neurons are fatigable, namely, the neurons
which have become a winner can not become a winner during acertain period of time. Because of this feature, FSOM tends toself-organize only in the area where input data are concentrated.
We investigate the behavior of FSOM and apply FSOM toclustering problems. Further, we introduce the fatigue level to
FSOM to increase its flexibility for various kinds of clustering
problems. The efficiencies of FSOM and the fatigue level are
confirmed by several simulation results.

I. I NTRODUCTION
In data mining, clustering is one of the most important
issues and is also very useful for many applications, suchas industrial engineering, image processing, biology andmedicine. Then, the Self-Organizing Map (SOM) attracts
attentions for clustering in recent years. SOM is popular tools
for clustering and visualization of high-dimensional data [1].
SOM is unsupervised neural network introduced by Kohonen
in 1982 [2] and is a model simplifying self-organization
process of the brain. SOM maps multidimensional data
onto a 2-dimensional grid. SOM can classify input dataaccording to similarities and patterns which are obtained by
the distance between neurons and is applied to wide fields
of data classifications. Although many methods to extract
clusters by using SOM have been proposed [3]-[10], it seems
to be very difficult to construct a simple method using SOM
for universal input data. On the one hand, in the world, theamount and complexity of data increase from year to year.
Therefore, it is important to investigate various extraction
methods of clusters from data including a lot of noises.
In this study, we propose a new type of SOM algorithm,
which is called Fatigable SOM (FSOM) algorithm. The
important feature of FSOM is that the neurons of FSOM
are fatigable, namely, in the learning process of FSOM,
the neurons which have become a winner can not becomea winner during a certain period of time. Because of thisfeature, FSOM tends to self-organize the area where theinput data are concentrated. We investigate the behavior of
FSOM and apply FSOM to clustering problems. Further, we
introduce the fatigue level to FSOM to increase the flexibility
Masato Tomita, Haruna Matsushita and Yoshifumi Nishio are with the
Department of Electrical and Electronic Engineering, Tokushima University,
Tokushima, Japan (phone: +81-88-656-7470; fax: +81-88-656-7471; email:tomita, haruna, nishio@ee.tokushima-u.ac.jp).

0-7803-9490-9/06/$20.00/©2006 IEEEfor various kinds of clustering problems. The efficiencies
of FSOM and the fatigue level are confirmed by several
simulation results.
In the Section II, the algorithm of the conventional SOM
is introduced. In the Section III, we explain the learningalgorithm of the proposed FSOM algorithm. In the Section
IV, the behavior of FSOM is explained with some simulation
results of clustering. In the Section V, we introduce thefatigue level to FSOM. In the Section VI, the efficiency of
the fatigue level is confirmed with some simulation results.
II. S ELF -O RGANIZING M AP (SOM)
In this section we introduce the conventional SOM algorithm in order to make clear new points of the proposed
algorithm.
SOM has two-layer structure of the input layer and theneurons arecompetitive layer. In the competitive layer,arranged as a regular 2-dimensional grid. The range of theelements of -dimensional input data Ü = ( ½ ¾)) are assumed to be from 0 to 1.
(     (SOM1) The initial values of all the weight vectors
 ½ ¾ of the neurons  
are given between 0 and 1 at random.
(SOM2) An input data Ü is inputted to all the neurons at
the same time in parallel.
(SOM3) We find the winner neuron by calculating the
distances between Ü and Û according to;Û   



 


  Û  Ü (1)In other words, the winner neuron is the neuron with theweight vector nearest to the input vector Ü . In this study,
Euclidean distance is used for Eq. (1).
(SOM4) The weight vectors of all the neurons are updatedas;Û   Û     Ü  Û  (2)where is the learning step. The function    is called
the neighborhood function and it is described as follows;




 



Ö

  

Ö  ¾

 ¾ 



(3)

where   is the learning rate, Ö and Ö  are the vectorial
locations of the neurons on the display grid, and  corresponds to the widths of the neighborhood function.
  and   decrease with time according to the following

6558

equations;        






(4)



where is the maximum number of the learning.
(SOM5) The steps from (SOM2) to (SOM4) are repeated for to.
all the input data, namely, from III. FATIGABLE SOM (FSOM)In this study, we propose Fatigable SOM (FSOM) algorithm. The important feature of FSOM is that the neurons
are fatigable. Namely, in the FSOM algorithm, the neurons
which have become a winner can not become a winner duringa certain period of time. We call the neuron which has
become a winner the fatigued neuron. We have very little
difficulty in understanding why neurons are fatigued withbecoming a winner. Because human-beings are fatigued withdoing something, neurons also are fatigued.
A. Learning Algorithm
We explain the learning algorithm of FSOM. In FSOM,
neurons are arranged as a regular 2-dimensional grid.
The range of the elements of -dimensional input data Ü= ( ½ ¾
) are assumed to be) (from 0 to 1.

     



(FSOM1) The initial values of all the weight vectors Û   =(  ½  ¾
) of the neurons are  ) (given between 0 and 1 at random.
(FSOM2) An input data Ü is inputted to all the neurons at
the same time in parallel.
(FSOM3) We find the winner neuron among all the neurons
EXCEPT the neurons which have become winners (namely,
the fatigued neurons) by calculating the distance between Üand Û  according to;   



 



 Û   Ü is the maximum number of the learning. The
where
learning function    is explained in the next subsection.
is labeled as a fatigued(FSOM5) The winner neuron
neuron.
(FSOM6) If FSOM has  fatigued neurons, all the fatigued
neurons are recovered and turn to be normal neurons.
(FSOM7) The steps from (FSOM2) to (FSOM6) are repeated to.
for all the input data, namely, from Û   

 

 Ü





 

     ÖWe use the learning function proposed in our past
study [10]. The value of the learning function is determinedby the distance between the input vector Ü and the weightvector Û   of the winner neuron according to;     Û   





 



 ¾



(9)

where  corresponds to the width of the learning function,and   corresponds to the maximum value of the learning
function.   decrease with time according to the following
equation;    
(10)











IV. S IMULATION R ESULTS OF FSOM
A. Behavior of FSOM
We consider the 2-dimensional input data of 400 points
whose distribution is non-uniform as Fig. 1(a). 200 points are
distributed within a small range from 0.3 to 0.5 horizontallyand from 0.3 to 0.5 vertically. The remaining 200 points
are uniformly distributed between 0 and 1 at random. We
consider the conventional SOM and the proposed FSOM with
100 neurons   . The parameters for the learning of
SOM and FSOM are chosen as follows;        


(6)





(7)

where    is the learning function, Ö  and Ö  are thevectorial locations of the neurons on the display grid, and  corresponds to the widths of the neighborhood function.
  decrease with time according to the following equation;







Û   Ü ¾







(11)

(5)

Ö  ¾

 ¾ 

 

B. Learning function

(8)

1

1

0.9

0.9

0.8

0.8

0.7

0.7

0.6

0.6

0.5

0.5

0.4

0.4

0.3

0.3

0.2

0.2

0.1

where is the learning step. The function     is called
the neighborhood function and it is described as follows;
  





In other words, the winner neuron is the neuron with theweight vector nearest to the input vector among the all
neurons except the fatigued neurons. In this study, Euclidean
distance is used for Eq. (5)(FSOM4) The weight vectors of all the neurons are updatedas;Û  



0
0

0.1
0.2

0.4

(a)
Fig. 1.
results.

0.6

0.8

1

0
0

0.2

0.4

0.6

0.8

1

(b)

Conventional SOM. (a) 2-dimensional input data. (b) Learning

We execute the learning for all input data once. The
simulation results of the conventional SOM is shown in
Fig. 1(b). We can see that the conventional SOM selforganizes all over the input data. Figure 2 shows the learning
process of the proposed FSOM. The final result is shown in
Fig. 2(f). FSOM self-organizes only the area where the input

6559

1

1

1

0.9

0.9

0.9

0.8

0.8

0.8

0.7

0.7

0.7

0.6

0.6

0.6

0.5

0.5

0.5

0.4

0.4

0.4

0.3

0.3

0.3

0.2

0.2

0.2

0.1

0.1

0
0

0.2

0.4

0.6

0.8

1

0
0

0.1
0.2

0.4

(a)

0.8

1

0
0

1

1

1

0.9

0.9

0.8

0.8

0.8

0.7

0.7

0.7

0.6

0.6

0.6

0.5

0.5

0.5

0.4

0.4

0.4

0.3

0.3

0.3

0.2

0.2

0.2

0.1

0.1

0.1

0
0

0
0

0.4

0.6

0.4

0.8

1

0.2

0.4

0.6

0.8

1

0
0

0.2

0.4

(e)

Learning of proposed FSOM. (a) Initial state ( = 0). (b)(Early behavior) In the early stage of the learning process,all the neurons of FSOM gather in the area where the input
data are concentrated as shown in Figs. 2(a) to (d). Thisbehavior can be explained by the most important feature of
FSOM, namely the neurons which have become a winner
can not become a winner during a certain period of time asshown in Fig. 3. When we use the conventional SOM, it ishighly possible that the same neurons become the winnermany times for the concentrated input data. This prevents
the neurons except the winner neurons from gathering in acluster. However, neurons in FSOM are fatigable and cannot
be a winner twice. This means that many different neurons
can be winners for the concentrated input data.
(Later behavior) In the later stage of the learning process, all
the neurons of FSOM move to fit the area where the input
data are concentrated as shown in Figs. 2(e) and (f). Thisbehavior can be explained by the learning function Eq. (9).
This learning function enhances the movement of the neurons
if the winner neuron is close to the input data. Hence, FSOM
tends to move to fit the area. On the other hand, this learning
function prevents the neurons to make a large move for input
data far apart from the neurons in the later stage. Hence,
FSOM does not spread out of the area where the input data
are concentrated.

0.8

1

0.6

0.8

1

(f)

= 5. (c)data are concentrated. Let us examine the behavior of FSOMin more detail.

0.6

(c)

0.9

0.2

0.2

(b)

(d)
Fig. 2.

0.6= 10. (d)= 20. (e)= 100. (f) Learning results ( = 400).

to self-organize only the area where the input data areconcentrated.
B. Application to Clustering
The concept using FSOM can be exploited to extract thedata only in clusters of the input data including a lot ofnoises, because FSOM can find such areas by itself. We carryout the extraction of cluster after Fig. 1(b) and Fig. 2(f). The
extraction method is a relatively simple as follows. Afterlearning, the input data which is within a radius of fromall neurons on the map are classified into the cluster.
The simulation results of the conventional SOM and
FSOM are shown in Figs. 4(a) and (b), respectively ( =0.05). As we can see from these figures, the cluster obtainedby the conventional SOM includes a lot of noises. However,
FSOM can successfully extract only the cluster.





1

1

0.9

0.9

0.8

0.8

0.7

0.7

0.6

0.6

0.5

0.5

0.4

0.4

0.3

0.3

0.2

0.2

0.1
0
0

0.1
0.2

0.4

(a)

0.6

0.8

1

0
0

0.2

0.4

0.6

0.8

1

(b)

Fig. 4. Extraction of cluster. (a) Cluster extracted by conventional SOM.
(b) Cluster extracted by FSOM.

Because of these two different features, FSOM tends

6560

(a)

(b)

(c)
Fig. 3.

How to determine a winner neuron . (a)V. FSOM WITH FATIGUE L EVEL
In this section, in order to increase the flexibility of FSOM,
we introduce a fatigue level to FSOM. The basic algorithm
is the same as the FSOM learning algorithm and we add an
additional index controlling update of the weight vectors.
) indicating the
We assign a variable  (fatigue level to each neuron of FSOM. This fatigue level is updated according to the following rule. We set theinitial value of  as 3.0. If a neuron becomes a winner, decreases  . Further, of all the neurons decrease
according to;

 







 




         Ü Û   (12)The value of  is reset to 3.0 when the fatigued neurons are






 







Û 



 Û         ÜÛ    (13)instead of Eq. (6).
(Case 2) For   :
The weight vector of neuron

 is updated according to;Û   Û   Ü Û   (14)




 





= 2. (c)= 3.

which is the same update equation as Eq. (6).
(Case 3) For   :
The weight vector of neuron is updated according to;



Û 



 Û         ÜÛ    (15)instead of Eq. (6).
(Case 4) For   :
The weight vector of neuron is updated according to Eq.
(14), where the initial values of and are reset to =0.7
and =4.0.









VI. S IMULATION R ESULTS OF FSOM WITH FATIGUE
L EVEL

recovered by (FSOM6).
The fatigue level influences the update of the weightvectors. This rule is added to realize the movement suchthat the neurons with lower fatigue levels move slower.
(Case 1) For   :
The weight vector of neuron is updated according to;= 1. (b)In this section, we explain the behavior of FSOM with thefatigue level with some simulation results.
A. Example 1
Input data is 2-dimensional random data of 400 points
whose distribution is non-uniform as Fig. 5. 200 points are
distributed within a small range from 0.2 to 0.4 horizontallyand from 0.2 to 0.4 vertically. The remaining 200 points are
uniformly distributed within a small range from 0.6 to 0.8
horizontally and from 0.6 to 0.8 vertically. The parameters of
SOM are the same as those in the simulation in the SectionIV.
We execute the learning for all input data once. The
simulation results of FSOM and FSOM with the fatigue level

6561

1

1

0.9

0.9

0.8

0.8

0.7

0.7

0.6

0.6

0.5

0.5

0.4

0.4

0.3

0.3

0.2

0.2

0.1

0.1

0
0

Fig. 5.

0.2

0.4

0.6

0.8

0
0

1

Fig. 7.

2-dimensional input data for Example 1.

are shown in Figs. 6(a) and (b), respectively. From the results,
we explain the difference between FSOM and FSOM withthe fatigue level. In the Section IV, we can see that FSOM
successfully extracts only cluster. However, when input datais containing two similar clusters as Fig. 5, FSOM spreadover the two clusters. However, by introducing the fatigue
level, the neurons gather in one of the two clusters.
1

1

0.9

0.9

0.8

0.8

0.7

0.7

0.6

0.6

0.5

0.5

0.4

0.4

0.3

0.3

0.2

0.2

0.1

0.1

0
0

0.2

0.4

(a)
Fig. 6.

0.6

0.8

1

0
0

0.2

0.4

0.6

0.8

1

2-dimensional input data for Example 2.

is reset according to (Case 4). This means that the fatigued
neurons in the cluster tend to tug other neurons. When pluralclusters exist, the cluster in which more fatigued neurons
exist can win the tug-of-war. Consequently, all of the neurons
gather to the cluster where the number of the input data is
the largest.
C. Application to Clustering
We carry out the extraction of cluster after Figs. 9(a), (b)and (c). The extraction method is the same as the Section
IV.
The simulation results of the conventional SOM, FSOM
and FSOM with the fatigue level are shown in Figs. 9(a),(b) and (c), respectively ( = 0.05). As we can see fromthe figures, the cluster obtained by the conventional SOM
includes a lot of noises. In addition, FSOM obtains two clusters including similar number of input data and a few noises
between two clusters. However, FSOM with the fatigue levelcan successfully extract only one cluster including the most
input data. As a result, by using FSOM with the fatigue level
for application to clustering, we can attained good results.
The algorithm of FSOM with the fatigue level is a little bitcomplicated and it may be difficult to find good parameters
for various kinds of input data. However, we consider that
this can increase the flexibility of FSOM.



0.2

0.4

0.6

0.8

1

(b)

Learning results. (a) FSOM. (b) FSOM with the fatigue level.

B. Example 2
Next, we consider more complicated case that input data is
2-dimensional random data of 800 points whose distribution
is non-uniform as Fig. 7. 320 points are distributed withina small range from 0.4 to 0.7 horizontally and from 0.6 to
0.75 vertically. 240 points are distributed within a small range
from 0.7 to 0.85 horizontally and from 0.25 to 0.4 vertically.
160 points are distributed within a small range from 0.15
to 0.35 horizontally and from 0.2 to 0.35 vertically. The
remaining 80 points are uniformly distributed between 0 and
1 at random. The parameters of SOM are the same as thosein the simulation in the Section IV.
We execute the learning for all input data once. The
simulation results are shown in Fig. 8. From the results, we
can say that the conventional SOM tends to self-organize
all the input data including a lot of noises and FSOM selforganizes some of the clusters. On the other hand, FSOM
with the fatigue level self-organizes only one cluster and is
not influenced by noises.
Because of the fatigue level, many neurons in the cluster
become fatigued neurons easily. Further, the learning function of the neurons whose fatigue levels become negative

VII. C ONCLUSIONSIn this study, we have proposed the Fatigable SOM(FSOM). We have explained the differences between SOM
and FSOM and FSOM with the fatigue level with learningalgorithm and have investigated its behavior. Furthermore,
we have applied the proposed FSOM to extract only one ofsome clusters including a lot of noises and have confirmed
its efficiency.
In the future, we try to discover new applications of FSOM
in diverse fields such as sound data processing.
R EFERENCES[1] T. Kohonen, “The Self-Organizing Maps,” Neurocomputing, vol. 21,
pp. 1-6, 1998.
[2] T. Kohonen, Self-Organizing Maps, Berlin, Springer, vol. 30, 1995.
[3] Y. Cheng, “Clustering with Competing Self-Organizing Maps,” Proc.
of IJCNN’92, vol. IV, pp. 785-790, 1992.

6562

1

1

1

0.9

0.9

0.9

0.8

0.8

0.8

0.7

0.7

0.7

0.6

0.6

0.6

0.5

0.5

0.5

0.4

0.4

0.4

0.3

0.3

0.3

0.2

0.2

0.2

0.1

0.1

0.1

0
0

0
0

0.2

0.4

0.6

0.8

1

0.2

0.4

(a)
Fig. 8.

0.6

0.8

1

0
0

1

1

0.9

0.9

0.8

0.8

0.8

0.7

0.7

0.7

0.6

0.6

0.6

0.5

0.5

0.5

0.4

0.4

0.4

0.3

0.3

0.3

0.2

0.2

0.2

0.1

0.1
0.4

0.6

0.8

1

0
0

0.8

1

0.1
0.2

0.4

(a)
Fig. 9.

0.6

Learning results. (a) Conventional SOM. (b) FSOM. (c) FSOM with the fatigue level.

1

0.2

0.4

(c)

0.9

0
0

0.2

(b)

0.6

(b)

0.8

1

0
0

0.2

0.4

0.6

0.8

(c)

Extraction of clusters. (a) Conventional SOM. (b) FSOM. (c) FSOM with the fatigue level.

[4] W. Wan and D. Fraser, “M2dSOMAP: Clustering and Classificationof Remotely Sensed Imagery by Combining Multiple Kohonen SelfOrganizing Maps and Associative Memory,” Proc. of IJCNN’93, vol. III,
pp. 2464-2467, 1993.
[5] J. Vesanto and E. Alhoniemi, “Clustering of the Self-Organizing Map,”
IEEE Transactions on Neural Networks, vol. 11, no. 3, pp. 586-600,
2000.
[6] C. Derek and E. Adrian, “Finding Curvilinear Features in Spatial Point
Patterns: Principal Curve Clustering with Noise,” IEEE Transactions on
Pattern Analysis and Machine Intelligence, vol. 22, no. 6, pp. 601-609,
2000.
[7] P. Doucette, P. Agouris and A. Stefanidis, “Self-Organized Clusteringfor Road Extraction in Classified Imagery,” ISPRS Journal of Photogrammetry and Remote Sensing, vol. 55, Issues 5-6, pp. 347-358,
2001.
[8] M. Gervey and S. Lek, “Patterning and Clustering Ecological Assemblages,” Proc. of WSOM’05, pp. 9-16, 2005.
[9] H. Matsushita and Y. Nishio, “Competing Behavior of Two Kinds of
SOMs and its Application to Clustering,” Proc. of WSOM’05, pp. 355362, 2005.
[10] H. Matsushita and Y. Nishio, “Competing and Accommodating Behaviors of Peace SOM,” Proc. of ISCAS’06, 2006 (to appear).

6563

1