2009 International Workshop on Nonlinear Circuits and Signal Processing
NCSP'09, Waikiki, Hawaii, March 1-3, 2009

Fuzzy ART Combining Overlapped Categories Using Variable Vigilance Parameters
Haruka Isawa, Haruna Matsushita and Yoshifumi Nishio
Tokushima University
2-1 Minami-Josanjima, Tokushima, Japan
Phone:+81-88-656-7470, Fax:+81-88-656-7471
Email: {isawa, haruna, nishio}@ee.tokushima-u.ac.jp

Abstractgory proliferation problem.
We have also proposed a Fuzzy ART Combining OverThe Fuzzy Adaptive Resonance Theory (Fuzzy ART) is an lapped Category in Consideration of Connections (Cunsupervised neural network and allows both binary and con- FART) [5]. C-FART makes connections between similar cattinuous input patterns. In this study, we propose a Fuzzy egories or releases connections at each step as FART-GL and
ART Combining Overlapped Categories Using Variable Vig- combines overlapping categories by using created connecilance Parameters. The vigilance parameters of the proposed tions. C-FART can reduce category proliferation by combinmethod are arranged for every category, and they are varied ing the categories with due consideration of their similarity.
according to the size of respective categories with learning. However, because the vigilance parameter is constant value
We confirm that the proposed Fuzzy ART can classify input although the categories become larger by combining overdata more flexible than the conventional Fuzzy ART.
lapped categories, a new category is created inside existing
categories. This creates a lot of overlapped categories.
1. Introduction
In this study, we improve the conventional C-FART andpropose a Fuzzy ART combining overlapped categories using
Self-organized clustering is a powerful tool whenever huge variable vigilance parameters. The proposed method has bothsets of data have to be divided into separate categories. In FART-GL and C-FART abilities. Moreover, the vigilance pathe field of neural network, the Adaptive Resonance Theory rameters are arranged for every category, and they are varied(ART) [1] is a popular representative for self-organized clus- according to the size of respective categories with learning.
tering. This theory has evolved as a series of real-time neural We investigate the behaviors of the proposed method by apnetwork models, that perform unsupervised and supervised plying to various input data.
learning, pattern recognition, and prediction. These models
are capable of learning stable recognition categories in response to arbitrary input sequences. The Fuzzy ART [2]- 2. Proposed Fuzzy ART[3] is the merger of fuzzy logic and ART neural network.
Fuzzy ART is applied to association, clustering and memThe structure of the proposed Fuzzy ART is same as theory of input pattern, and it classifies the input data into each standard Fuzzy ART.
appropriate category by creating rectangles. However, be- Input vector: Each input I is an m-dimensional vector I =cause Fuzzy ART frequently classifies the input data of the (I1 , · · · , Im ), where Ii ∈ [0, 1]m .
common categories into several categories, a category prolif- Weight vector: Each category j corresponds to a vector
eration problem occurs. For this reason, Fuzzy ART perfor- w j = (w j1 , · · · , w jm ), ( j = 1, · · · , n) of an adaptive weight.
mance is highly dependent on a vigilance parameter which The number of potential categories n is arbitrary. Initiallycontrols category size.
w j1 = · · · = w jm = 1.
To solve these problems, we have proposed two types of Parameters: The proposed Fuzzy ART dynamics are deFuzzy ART in our past studies. The first is Fuzzy ART with termined by a choice parameter α > 0; a learning paramGroup Learning (FART-GL) [4]. FART-GL makes connec- eter β ∈ [0, 1]; and the vigilance parameter ρ j ∈ [0, 1].
tions between categories or releases connection as human re- In contrast to the standard Fuzzy ART and the conventionallationships which keep changing with time in the real world. C-FART whose vigilance parameter is a fixed value for all
The connection is created between similar categories, and the categories, the vigilance parameters of the proposed method
connected categories are learned as “group” of category. By are arranged for every category and are varied according
using this method, the input data are classified into each ap- to the size of respective categories with learning. Initiallypropriate group. However, FART-GL can not solve the cate- ρ1 = · · · = ρn = ρ0 .

- 661 -

Connection: The proposed method has a connection C andthe age of the connections age. Both C and age are n × n matrices. The initial values of C and age are set to zero. If thecategories J and j are connected with learning, C J, j changes
from zero to one.
2.1. Learning Algorithm of Proposed Method

Process I : Learning(StepI-1) An input vector I is inputted to the category layerfrom the input layer.
(StepI-2) We calculate choice function T j of each category j;
T j (I) =| I ∧ wj |,(α+ | w j |)(1)

where the fuzzy AND [6] operator ∧ and the norm | · | aredefined by(p ∧ q)i ≡ min(pi , qi ), | P |≡m∑

| pi |.

0.6

(2)

i=1

0.3

j

(3)

(4)

If Eq. (4) is not satisfied, a new index J is chosen by Eq. (3),and the search process continues until the chosen J satisfies
Eq. (4).
(StepI-4) If any J satisfies Eq. (4), w J is updated byoldwnew= β(I ∧ woldJ ) + (1 − β)w J ,
J

0.1

uJ
0.4 0.5(5)

and we perform (StepII-1). On the contrary, If all categories
do not satisfy Eq. (4), a new category is established wn+1 = I,
and we proceed to next input data.

uJnew
0.2

0.7

w J = (uJ , v Jc ) = (0.2, 0.1, 0.5, 0.6)
w k = (u k , v ) = (0.4, 0.3, 0.3, 0.4)
c
k

(a)

0.7
c new

w Jnew = (uJnew , v J)= (0.2, 0.1, 0.3, 0.4)(b)Figure 1: Combining process of overlapped categories.
(a) Category J and category k are overlapping. (b) If C J,k = 1,
w J are combined according to Eq. (9), and k is removed. The
vigilance parameter of the proposed method is varied according to (StepIV-1).

Process II : Update Connections(StepII-1) If J2 does not exist, we skip this step and perform(StepII-2). The similarity of I and w J2 is measured as Eq. (4).
If its similarity satisfies ρ J2 , a connection between the winning category J and the second-winning category J2 is created as C J,J2 = 1. The age of the connection between J andJ2 is set to zero (“refresh” the age) age J,J2 = 0. If Eq. (4) is
not satisfied, the connection is not updated.
(StepII-2) The age of all categories, which directly connect
with the winning category J, are increased one;old
agenew
J, j = age J, j + 1,

If more than one T j is maximal, the category j with the smallest index is chosen as the winning category J.
A second-winning category J2 is found for updating connections, if J2 exists.
(StepI-3) The similarity of I and w J is measured by the vigilance criterion according to| I ∧ wJ |≥ ρJ .
|I|

uk

0.2

The winning category J, whose T j is maximum, is found;
J = arg max{T j }.

v Jnew

0.6

vJ

0.4

0.1

The learning algorithm of the proposed method consists ofmainly four processes: 1) Learning, 2) Update Connections,
3) Combining Categories and 4) Varying the vigilance parameter. These four steps are repeated for all the input data set.
Therefore, the proposed method makes or releases connections at each step, and overlapped categories are combinedwith considering their connections. Furthermore, the vigilance parameter varies according to the size of the combined
category.

vkj ∈ NJ ,(6)

where N J is the set of categories which directly connect withJ, namely C J, j = 1.
(StepII-3) The connections are removed, if their age exceeds
a threshold value AT (t);C J, j = 0,
where

i f age J, j ≥ AT (t),

(7)

(
) t
AT (t) = AT i AT f /AT i tmax ,(8)where t is the learning step, tmax is the learning length, AT i
and AT f is the initial and final values of AT , respectively.
Process III : Combining Categories(StepIII-1) We check whether the winning category J is overlapping with other category k and combine these categories.
The input data belonging to the category k overlapping withthe category J are classified to J, and the category k is removed, therefore, nnew = nold − 1.
Without loss of generality, we assume the weight vector w j
can be written in complement cording form [6]: w j = (u j , vcj ),where u j and vcj are m-dimensional vectors. We find the- 662 -

1

1

1

1

0.9

0.9

0.9

0.9

0.8

0.8

0.8

0.8

0.7

0.7

0.7

0.7

0.6

0.6

0.6

0.6

0.5

0.5

0.5

0.5

0.4

0.4

0.4

0.4

0.3

0.3

0.3

0.3

0.2

0.2

0.2

0.2

0.1

0.1

0.1

0.1

0
0

0
0

0
0

0.2

0.4

(a)

0.6

0.8

1

0.2

0.4

0.6

0.8

1

0.2

(b)

0.4

(c)

0.6

0.8

1

0
0

0.2

0.4

0.6

0.8

1

(d)

Figure 2: Simulation 1 for rectangular-shaped data. (a) Input data. (b) Simulation result of Fuzzy ART. (c) Simulation resultof C-FART. (d) Simulation result of the proposed method.

category k overlapping with J as Fig. 1(a), in other words,
Fig. 1(a) is satisfied for any dimension i.
Let vector u J and vk define one corner of a rectangle, andlet v J and vk define another corner of rectangle as Fig. 1(a). If
k exists and directly connects with J, namely C J,k = 1, J andk are combined as Fig. 1(b) by{)}(w J new = (u J ∧ uk ) , vcJ ∧ vck .
(9)

Table 1: Number of categories of Simulation 1.
Number of categories
Algorithm
Minimum Maximum Average
Fuzzy ART
47
59
52.9
C-FART
39
54
45.7
Proposed method
17
46
29.8

input data is 450 points, the inside cluster has 100 points andthe outside cluster has 350 points. The input data are sorted
at random. When t = 0, there is no category, namely n = 0.
1− | w J |The parameters for the learning are chosen as follows;
ρJ =
,
(10)
(For Fuzzy ART) α = 0.1, β = 1.0, ρ = 0.9,m(For C-FART) α = 0.1, β = 1.0, ρ = 0.9, AT i = 2, AT f = 25
In the standard Fuzzy ART, the vigilance parameter is in(For proposed method) α = 0.1, β = 1.0, ρ0 = 0.9, AT i =variant. However, the vigilance parameter of the proposed
2, AT f = 25.
method is varied according to the size of combined category.
The learning results of the Fuzzy ART and C-FART are
shown in Figs. 2(b) and (c), respectively. We can see that
3. Computer Simulations
the category proliferation occurs, and there are a lot of overWe apply the proposed method to 2-dimensional input lapped area. On the other hand, the result of the proposed
data, and compare the proposed method with the standard method is shown in Fig. 2(d). From this figure, we can see
Fuzzy ART and conventional C-FART. Preferable features of that the category proliferation and the overlapped area are reduced. This is because that the vigilance parameter is varthe obtained results are listed below;
ied according to the size of a category with learning. There1. High classification accuracy.
fore, the proposed method can categorize the input data more
2. The number of categories is small.
flexible than other two methods. Furthermore, the proposed
3. Overlapped area are small.
Without high classification accuracy, the input data belonging method can recognize the group of the categories by whether
to diﬀerent cluster into same category. If there are many cat- there are connections between categories, and its results are
egories, information of categorization is too much. With high shown in diﬀerent colors. Therefore, we can obtain not onlyclassification accuracy, it is preferable that overlapped area is the detailed informations by the categorization results, butalso the relationships between categories by the groups.
small.
Furthermore, we carry out the learning simulations repeated 100 times, and Table 1 summarizes performances with
3.1. Simulation 1
the minimum, maximum and average values of the numberFirst, we consider 2-dimensional rectangular-shaped input of categories over 100 independent runs. The minimum anddata as Fig. 2(a), consisting of 2-clusters. Total number of the the maximum values mean best and worst results in 100 simProcess IV : Varying the vigilance parameter(StepIV-1) The vigilance parameter ρ J is varied by- 663 -

1

1

1

1

0.9

0.9

0.9

0.9

0.8

0.8

0.8

0.8

0.7

0.7

0.7

0.7

0.6

0.6

0.6

0.6

0.5

0.5

0.5

0.5

0.4

0.4

0.4

0.4

0.3

0.3

0.3

0.3

0.2

0.2

0.2

0.2

0.1

0.1

0.1

0.1

0
0

0
0

0
0

0.2

0.4

0.6

(a)

0.8

1

0.2

0.4

0.6

0.8

1

0.2

(b)

0.4

0.6

0.8

1

0
0

(c)

0.2

0.4

0.6

0.8

1

(d)

Figure 3: Simulation 2 for circular-shaped data. (a) Input data. (b) Simulation result of Fuzzy ART. (c) Simulation result of
C-FART. (d) Simulation result of the proposed method.

ulations, respectively. All the number of categories of the
proposed method are smaller than Fuzzy ART and C-FART.
From the average values, the proposed method has reduced
category proliferation 43% and 34% from Fuzzy ART and
C-FART, respectively. We can obtain more eﬀective information of categorization than Fuzzy ART and C-FART because
the proposed method can reduce the number of categories.

Table 2: Number of categories of Simulation 2.
Number of categories
Algorithm
Minimum Maximum Average
Fuzzy ART
43
55
48.4
C-FART
32
51
40.4
Proposed method
14
41
26.4

3.2. Simulation 2
Next, we apply the proposed method to circular-shaped input data, which have 2-clusters as Fig. 3(a). The inside cluster
has 70 points and the outside cluster has 500 points. It is diﬃcult to classify the input data such as Fig. 3(a) into appropriate
categories because categories are created by rectangles. The
learning conditions are the same used in Simulation 1.
Figures 3(b) and (c) show the simulation results of Fuzzy
ART and C-FART, respectively. We can see that category
proliferation occurs, and there are a lot of overlapped area.
On the other side, in the results of the proposed method shownin Fig. 3(d), the category proliferation and the overlapped area
are reduced. Furthermore, we can recognize the group of thecategories from its color from the result of categorization.
The performances with the minimum, maximum and average number of categories over 100 independent runs arelisted in Table 2. We can see that all the number of categories of the proposed method are smaller than Fuzzy ART
and C-FART. From the average values, the proposed method
has reduced category proliferation 45% and 35% from Fuzzy
ART and C-FART, respectively. Therefore, we have obtained
more eﬀective information of categorization than Fuzzy ART
and C-FART by using the proposed method.
4. Conclusions
In this study, we have proposed the new type of Fuzzy
ART. As important features of the proposed method, the vigilance parameter are arranged for every category, and they arevaried in accordance with the size of category with learning.
We have applied the proposed method into 2-dimensional input data, and the learning behaviors of the proposed method
have been investigated. We have confirmed that the proposed
method can categorize the input data more flexible than thestandard Fuzzy ART and the conventional C-FART, and have
reduced category the proliferation and the overlapped area.
References[1] C.A. Carpenter, “Distributed learning, recognition, and prediction by ART and ARTMAP neural networks,” Neural Networks,
vol. 10, pp. 1473–1494, 1997.
[2] G.A. Carpenter, S. Grossberg, D.B. Rosen, “Fuzzy ART: Fast
stable learning and categorization of analog patterns by an adaptive resonance system,” Neural Networks, vol. 4, pp. 759–771,
1991.
[3] T. Frank, K.F. Kraiss and T. Kuhlen, “Competitive analysis
of Fuzzy ART and ART-2A network clustering performance,”
IEEE Trans. Neural Networks, vol. 9, no. 3, pp. 544–559, 1998.
[4] H. Isawa, M. Tomita, H. Matsushita and Y. Nishio, “Fuzzy
Adaptive Resonance Theory with Group Learning and its Applications,” Proc. of International Symposium on Nonlinear
Theory and its Applications, pp. 292-295, 2007.
[5] H. Isawa, H. Matsushita and Y. Nishio, “Fuzzy Adaptive Resonance Theory Combining Overlapped Category in Consideration of Connections,” Proc. of International Joint Conference
on Neural Networks, pp. 3594–3599, 2008.
[6] L. Zadeh, “Fuzzy sets,” Information and Control, vol. 8, pp.
338–353, 1965.

- 664 -