2008 IEEE Workshop on
Nonlinear Circuit Networks, Tokushima, Dec 5-6, 2008

Improved Fuzzy Adaptive Resonance Theory
Combining Overlapped Category
in Consideration of Connections
Haruka ISAWA, Haruna MATSUSHITA and Yoshifumi NISHIO
Dept. Electrical and Electronic Eng.,
Tokushima University
Email: {isawa, haruna, nishio}@ee.tokushima-u.ac.jp
Abstract— The Adaptive Resonance Theory (ART) is an unsupervised neural network. Fuzzy ART is a variation of ART, allows
both binary and continuous input patterns. However, Fuzzy ART
has the category proliferation problem. To solve this problem, wehave proposed Fuzzy ART Combining Overlapped Category inconsideration of connections (C-FART) in previous study. In thisstudy, we propose a improved C-FART. The important featuresof the improved C-FART are that the vigilance parameters arearranged for every category and they are varied according tothe size of respective categories with learning. We investigate thebehavior of the improved C-FART, and compare the improved CFART with the simple Fuzzy ART and the conventional C-FART.

I. I NTRODUCTION
Self-organized clustering is a powerful tool whenever hugesets of data have to be divided into separate categories. In
the field of neural network, the Adaptive Resonance Theory (ART) [1], introduced and developed G.A.Carpenter andS.Grossberg, is a popular representative for self-organizedclustering. Some outstanding features of ART, besides its
clustering capabilities, have attracted the attention from application engineers. This theory has evolved as a series of realtime neural network models that perform unsupervised andsupervised learning, pattern recognition, and prediction. Thesemodels are capable of learning stable recognition categories in
response to arbitrary input sequences. However, the fusion ofa computational efficiency of neural network and a capabilityof fuzzy logic to represent complex class boundaries, hascreated a lot of interest in neurofuzzy pattern recognition
systems. Then, we pay attentions Fuzzy ART [2], which is
the merger of fuzzy logic and ART neural network. Fuzzy
ART is applied to association, clustering and memory of input
pattern. Fuzzy ART classified input data into each appropriate
category by creating rectangles. However, Fuzzy ART has
category proliferation problem. For this reason, Fuzzy ARTperformance is highly dependant on a vigilance parameter,
which controls category size.
Then, we have noted overlapped category and have proposed
Fuzzy Adaptive Resonance Theory Combining Overlapped
Category in Consideration of Connections (C-FART) [5] inprevious study. C-FART has two important features. One is
to make connections between similar categories, and the otheris to combine overlapped categories into one category withconnections. With this method, we can reduce category proliferation. However, C-FART also has a disadvantage. Categories
get larger by combining overlapped categories as Fig. 1.
Therefore, if input data is inputted in the shaded area inFig. 1(b), it is recognized as a new category. In other words, a
new category is created inside a category. This is because thevigilance parameter is constant value. For all of these reasons,a lot of overlapped categories are created.
In this study, we propose improved C-FART. The importantfeatures of the improved C-FART are that the vigilance parameters are arranged for every category and they are varied
according to the size of respective categories with learning. Weinvestigate the behavior of the improved C-FART and compare
the improved C-FART with the simple Fuzzy ART and theconventional C-FART.
II. P ROPOSED F UZZY ART (T HE IMPROVED C-FART)
A. Structure of the improved C-FART
The structure of the improved C-FART is same as theconventional Fuzzy ART.
Input vector: Each input I is an m-dimensional vector I =(I1 , · · · , Im ), where Ii ∈ [0, 1]m .
Weight vector: Each category j corresponds to a vector
wj = (wj1 , · · · , wjm ), (j = 1, · · · , n) of an adaptive weight.
The number of potential categories n is arbitrary. Initially
wj1 = · · · = wjm = 1.
Parameters: The improved C-FART dynamics are determinedby a choice parameter α > 0; a learning parameter β ∈[0, 1]; and the vigilance parameter ρj ∈ [0, 1]. The vigilanceparameter is a fixed value for all categories in the simple Fuzzy
ART and the conventional C-FART. The important features of
the improved C-FART are that the vigilance parameters arearranged for every category and they are varied according
to the size of respective categories with learning. Initiallyρ1 = · · · = ρn = ρ0 .
Connection: The improved C-FART has a connection C andthe age of the connections age. Both C and age are n × nmatrices. The initial values of C and age are set to zero. If thecategories J and j are connected with learning, CJ,j changes
from zero to one.

-8-

B. Learning algorithm of the improved C-FART
The learning algorithm of the improved C-FART consistsof mainly four processes: 1) Learning, 2) Update Connections, 3) Combining Categories and 4) Varying the vigilanceparameter. These four steps are repeated for all input data set.
Therefore, the improved C-FART makes or releases connections at each step, and overlapped categories are combinedwith considering their connections. Furthermore, the vigilanceparameter varies according to the size of the combined category.
Process I : Learning(StepI-1) An input vector I is inputted to the category layer
F2 from the input layer F1 .
(StepI-2) For the input vector I and the category j, choicefunction Tj is defined byTj (I) =m∑| pi |.

(2)

i=1

The winning category J, whose Tj is maximum, is found;
J = arg max{Tj }.
j(3)If more than one Tj is maximal, the category j with the smallest index is chosen as the winning category J. Furthermore, asecond-winning category J2 , whose TJ2 is the second largest
next to TJ , is found for updating connections if J2 exists.
(StepI-3) The similarity of I and wJ is measured by thevigilance criterion according to| I ∧ wJ |≥ ρJ .
|I|(4)

If Eq. (4) is not satisfied, a new index J is chosen by Eq. (3)and the search process continues until the chosen J satisfies
Eq. (4).
(StepI-4) If any J satisfies Eq. (4), wJ is updated byoldwnew= β(I ∧ woldJJ ) + (1 − β)w J ,(5)and we perform (StepII-1). On the contrary, if all categories
do not satisfy Eq. (4), a new category is established;(6)

and we proceed to next input data.
Process II : Update Connections(StepII-1) If J2 does not exist, we skip this step and perform(StepII-2). The similarity of I and wJ2 is measured as Eq. (4).
If its similarity satisfies ρJ2 , a connection between the winningcategory J and the second-winning category J2 is created;
CJ,J2 = 1.

oldagenew
J,j = ageJ,j + 1,

(7)

j ∈ NJ ,(9)where NJ is the set of categories which directly connect withJ, namely CJ,j = 1.
(StepII-3) The connections are removed, if their age exceeds
a threshold value AT (t);CJ,j = 0,(1)

where the fuzzy AND [4] operator ∧ and the norm | · | aredefined by

wn+1 = I,

On the contrary, if Eq. (4) is not satisfied, the connection is
not updated.
(StepII-2) The age of all categories, which directly connect
with the winning category J, are increased one;if ageJ,j ≥ AT (t),where| I ∧ wj |,(α+ | wj |)(p ∧ q)i ≡ min(pi , qi ), | P |≡The age of the connection between J and J2 is set to zero(“refresh” the age);
ageJ,J2 = 0.
(8)

(AT (t) = ATi

ATfATi(10))t tmax,(11)

where t is the learning step, tmax is the learning length,
ATi and ATf is the initial value and the final value of AT ,respectively.
Process III : Combining Categories(StepIII-1) We check whether the winning category J isoverlapping with other category and combine these categories.
The inputs belonging to the category k are classified to thecategory J, and the category k is removed, therefore, nnew =nold − 1.
Without loss of generality, we assume the weight vector
wj can be written in complement cording form [4]: wj =(uj , v cj ), where uj and v cj are m-dimensional vectors. We
find the category k which are overlapping with J as Fig. 1(a),in other words, Fig. 1(a) is satisfied for any dimension i.
Let vector uJ and v k define one corner of a rectangle, andlet v J and v k define another corner of rectangle as Fig. 1(a).
If k exists and directly connects with J, namely CJ,k = 1, J
and k are combined as Fig. 1(b) bywJ new = {(uJ ∧ uk ) , (v cJ ∧ v ck )}.

(12)

Process IV : Varying the vigilance parameter(StepIV-1) The vigilance parameter ρJ is varied by
1− | wJ |,(13)m
In the conventional Fuzzy ART, the vigilance parameter isinvariant. However, the vigilance parameter is varied according
to the size of combined category in the improved C-FART.
ρJ =III. S IMULATION R ESULTS
We apply the improved C-FART to 2-dimensional input
data, and compare the improved C-FART with the simple
Fuzzy ART and the conventional C-FART.
We consider 2-dimensional input data as Fig. 2(a), consisting of 2-clusters. Total number of the input data is 450 points,-9-vk

0.6

vJ0.40.3
0.1v Jnew

0.6

uk

u0.1

uJ0.2

0.4 0.5

new
J

0.2

0.7

w J = (uJ , v ) = (0.2, 0.1, 0.5, 0.6)
cJw

newJw k = (u k , v kc ) = (0.4, 0.3, 0.3, 0.4)

(a)

0.7

= (unew
J,vc new
J)

0.8

0.8

0.7

0.7

0.6

0.6

0.5

0.5

0.4

0.4

0.3

0.3

0.2

0.2

0.1

0.1
0.2

0.4

0.6

0.8

1

000.2

(a)

= (0.2, 0.1, 0.3, 0.4)

Fig. 1. Overlapped categories combining process in the improved C-FART.
(a) Category J and category k are overlapping. (b) If CJ,k = 1, wJ are
combined according to Eq. 12, and k is removed. The vigilance parameter isconstant in the conventional Fuzzy ART, however the vigilance parameter isvaried at (StepIV-1) in the improved C-FART.

1

1

0.9

0.9

0.8

0.8

0.7

0.7

0.6

0.6

0.5

0.5

0.4

0.4

0.3

0.3

0.2

0.2

00N UMBER OF CATEGORIES

0.6

0.8

10.6

0.8

10.1
0.2

0.4

0.6

0.8

1

000.2

(c)Number of categories
Minimum Maximum
Average
4759
52.88
3954
45.66
1746
29.77

0.4

(b)

0.1

TABLE I

Fuzzy ART
C-FART
Improved CFART

10.900(b)Algorithm

10.90.4(d)Fig. 2. Simulation results. (a) Input data. (b) Simulation result of the simple
Fuzzy ART. (c) Simulation result of the conventional C-FART. (d) Simulation
result of the improved C-FART.

the inside cluster has 100 points and the outside cluster has
350 points. The input data are sorted at random. When learningstarts, there is no category, namely n = 0. The parameters forthe learning are chosen as follows;(For Fuzzy ART)α = 0.1, β = 1.0, ρ = 0.9,
(For the conventional C-FART)α = 0.1, β = 1.0, ρ = 0.9, ATi = 2, ATf = 7,
(For the improved C-FART)α = 0.1, β = 1.0, ρ0 = 0.9, ATi = 2, ATf = 7,

and the maximum values mean best and worst results in
100 simulations, respectively. All the number of categoriesof the improved C-FART are smaller than the simple Fuzzy
ART and the conventional C-FAT. From the average values,the improved C-FART has reduced category proliferation andoverlapped categories by 43% as compared to the simple
Fuzzy ART. Furthermore, the improved C-FART has improved
34% from using the conventional C-FART. We have obtained
more effective results than the simple Fuzzy ART and theconventional C-FART by using the improved C-FART.
IV. C ONCLUSIONSThe learning results of the simple Fuzzy ART and theconventional C-FART are shown in Figs. 2(b) and (c), respectively. We can see that category proliferation occurs and a lotof overlapped categories are created. The other side, the resultof the improved C-FART is shown in Fig. 2(d). From thisfigure, we can see that category proliferation and overlapped
categories are reduced. This is because the vigilance parameteris varied according to the size of category with learning. In
addition, the improved C-FART can classify input data into
more appropriate category because the vigilance parameter isarranged for all categories.
We perform the learning simulation 100 times to the input
data created in the same way as Fig. 2(a). The input data arecreated each time and are sorted by random to remove thedependence on the order of the input data.
Table I summarizes minimum, maximum and average valuesof the number of categories. In other words, the minimum

In this study, we have proposed the improved C-FART tosolve the category proliferation and overlapped category problem. The improved C-FART varies the vigilance parameterin accordance with the size of category. We have applied
the improved C-FART into 2-dimensional input data, andthe learning behaviors of the improved C-FART have been
investigated. We have confirmed that the improved C-FART
can obtain more effective result than the simple Fuzzy ART
and the conventional C-FART, and have reduced category
proliferation and overlapped categories.
R EFERENCES[1] C.A. Carpenter, “Distributed learning, recognition, and prediction by ART
and ARTMAP neural networks,” Neural Networks, vol. 10, pp. 1473–1494, 1997.
[2] G.A. Carpenter, S. Grossberg, D.B. Rosen, “Fuzzy ART: Fast stablelearning and categorization of analog patterns by an adaptive resonancesystem,” Neural Networks, vol. 4, pp. 759–771, 1991.

- 10 -[3] T. Frank, K.F. Kraiss and T. Kuhlen, “Competitive analysis of Fuzzy
ART and ART-2A network clustering performance,” IEEE Trans. NeuralNetworks, vol. 9, no. 3, pp. 544–559, 1998.
[4] L. Zadeh, “Fuzzy sets,” Information and Control, vol. 8, pp. 338–353,
1965.
[5] H. Isawa, H. Matsushita and Y. Nishio, “Fuzzy Adaptive Resonance Theory Combining Overlapped Category in Consideration of Connections”
Proc. of International Joint Conference on Neural Networks, pp. 3594–3599, 2008.

- 11 -