Fuzzy Adaptive Resonance Theory Combining Overlapped
Category in Consideration of Connections
Haruka Isawa, Haruna Matsushita and Yoshifumi Nishio

Abstract— Adaptive Resonance Theory (ART) is an unsupervised neural network. Fuzzy ART (FART) is a variationof ART, allows both binary and continuous input patterns.
However, Fuzzy ART has the category proliferation problem.
In this study, to solve this problem, we propose a new Fuzzy
ART algorithm: Fuzzy ART Combining Overlapped Category
in consideration of connections (C-FART). C-FART has two
important features. One is to make connections between similar
categories. The other is to combine overlapping categories intowith connections one category. We investigate the behavior of
C-FART, and compare C-FART with the conventional FART.

I. I NTRODUCTIONS

ELF-ORGANIZED clustering is a powerful tool whenever huge sets of data have to be divided into separate
categories. In the field of neural network, the Adaptive
Resonance Theory (ART), introduced and developed by G.A.
Carpenter and S. Grossberg [1], is a popular representative
for self-organized clustering. Some outstanding features ofART, besides its clustering capabilities, have attracted theattention from application engineers. This theory has evolved
as a series of real-time neural network models that perform
unsupervised and supervised learning, pattern recognition,and prediction. These models are capable of learning stable recognition categories in response to arbitrary input
sequences. However, the fusion of a computational efficiencyof neural network and a capability of fuzzy logic to represent
complex class boundaries, has created a lot of interest inneurofuzzy pattern recognition systems. Then, we pay attentions Fuzzy ART (FART) [2], which is the merger of fuzzy
logic and ART neural network. FART is an unsupervisedneural network and incorporates the basic features of all
ART systems [3]-[4]. Furthermore, by creating hyperboxes,
input vectors are classified into each appropriate category.
For this reason, FART often makes input data of the common
categories classify several categories, and FART performanceis highly dependant on a vigilance parameter, which controls hyperbox size. Therefore, FART causes the category
proliferation problem. In order to solve this problem, themethod considering overlapping categories is proposed in
hyperbox fuzzy set [6]. However, if all overlapping categories
are combined in FART, the input data are not classified into
each appropriate category since the hyperboxes keep getting
larger and larger.
Our past study proposed Fuzzy ART with Group Learning(FART-GL) [5]. FART-GL has connections between cateHaruka Isawa, Haruna Matsushita and Yoshifumi Nishio are with the
Department of Electrical and Electronic Engineering, Tokushima University,
Tokushima, 770–8506, Japan (phone: +81–88–656–7470; fax: +81–88–656–
7471; email: {haruka, haruna, nishio}@ee.tokushima-u.ac.jp).

gories as human relationship which keeps changing with timein the real world. The connection is created between similar
categories, and the categories, which have connections, arelearned as “group” of category. By using this method, sample
data are effectively classified into each appropriate group.
However, FART-GL can not reduce category proliferation.
In this study, we propose a Fuzzy ART Combining
Overlapped Category in consideration of connections (CFART) to solve the category proliferation problem. C-FART
makes connections between similar categories or releases
connections at each step, in addition, C-FART combines
the overlapping categories using the created connections.
In other words, C-FART combines the categories with due
consideration of their similarity.
In Section II, the algorithm of the conventional FART is
introduced. In Section III, we explain the learning algorithmof the proposed C-FART in detail. We apply C-FART tovarious input data sets, and the learning behaviors of C-FART
are investigated in Section IV. The learning performance of
C-FART is compared with FART. We confirm that the proposed C-FART can perform more effective learning than theconventional FART and can reduce the category proliferation
problem.
II. C ONVENTIONAL F UZZY ART
A. Structure of Fuzzy ART
Fuzzy ART is composed of F1 (input layer) and F2(category layer). F1 and F2 are connected by the bottomup-weight vector wij and the top-down-weight vector wji .
m neurons of the input layer F1 correspond to the an input
vector I.
Input vector: Each input I is an m-dimensional vector I =(I1 , I2 , · · · , Im ), where each component Ii (i = 1, · · · , m)
is in the interval [0, 1].
Weight vector: Each category j corresponds to a vector
wj = (wj1 , · · · , wjm ), (j = 1, · · · , n) of adaptive weight,or LTM (long-term-memory) traces. The number of potential
categories n is arbitrary. Initially
wj1 = · · · = wjm = 1.

Parameters: Fuzzy ART dynamics are determined bychoice parameter α > 0; learning parameter β ∈ [0, 1];
and vigilance parameter ρ ∈ [0, 1].
B. Learning Algorithm of Fuzzy ART
We explain the learning algorithm of the conventional
Fuzzy ART.

3594
c
978-1-4244-1821-3/08/$25.002008
IEEE

(1)

(FART1) An input vector I is inputted to the category layer
F2 from the input layer F1 .
(FART2) A winning category J is chosen. For the input
vector I and category j, choice function Tj is defined by
Tj (I) =| I ∧ wj |,(α+ | wj |)(2)

where the fuzzy AND [7] operator ∧ and the norm | · | aredefined by(p ∧ q)i ≡ min(pi , qi ),(3)m| pi |.
(4)| P |≡
The winning category J, whose Tj is maximum, is found;(5)

If more than one Tj is maximal, the category j with thesmallest index is chosen as the winner J.
(FART3) The similarity of I and the current winning category wJ is measured by the vigilance criterion. We check
whether| I ∧ wJ |
≥ ρ.
(6)
|I|
If Eq. (6) is not satisfied, namely| I ∧ wJ |
< ρ,
(7)
|I|
a new index J is chosen by Eq. (5). The search processcontinues until the chosen J satisfies Eq. (6).
(FART4) wJ is updated byold= β(I ∧ wold
wnewJJ ) + (1 − β)w J ,(8)

if there is J which satisfies Eq. (6). On the contrary, if allavailable F2 nodes do not satisfy Eq. (6), a new category isestablished in F2 ;wn+1 = I.
(9)(FART5) The steps from (FART1) to (FART4) are repeated
for all the input data.
C. Complement Coding
Because vector elements of prototype can only become
smaller by adaptation, a Fuzzy ART network tends to create
more and more prototype over time. This behavior is avoided
by normalizing input to constant vector length, this method
is called complement coding. The complement coded input
I to the recognition system is the 2m-dimensional vector;
where

I

0.4

0.2

uk
0.1

0.3

0.7

w k = (uk , v ) = (0.1, 0.2, 0.3, 0.4)ck

I = (a, a c ) = (0.3, 0.4, 0.7, 0.6)

i=1

TJ = max{Tj : j = 1 · · · n}.

vk

0.6

I = (a, ac ) ≡ (a1 , · · · , am , ac1 , · · · acm ).

(10)aci ≡ 1 − ai .

(11)

In this case, the weight vector wj can be written in complement coding form;
w j = (uj , v cj ) = (wj1 , · · · , wjm , wjm+1 , · · · , wj2m ),(12)
where uj and v j are m-dimensional vectors as uj =(wj1 , · · · , wjm ) and v j = (1 − wjm+1 , · · · , 1 − wj2m ).

Fig. 2. If I is fully contained by any category k, I is classified to thecategory k even if k is not the winning category.

III. P ROPOSED F UZZY ART (C-FART)In this study, we propose the combining overlapped category spaces Fuzzy ART considering connections (C-FART).
C-FART learning algorithm consists of mainly three steps:
1) Learning, 2) Update Connections and 3) Combining
Categories. A flowchart of the learning algorithm is shownin Fig. 1. As important features of C-FART, C-FART learns
connections between categories, and overlapped categories
are combined with considering their connections. Therefore,
if there is no connection between categories, overlapping
categories are not combined.
Connection: C-FART has a connection matrix denoted by C
and the age of the connections denoted by age. Both C andage are n × n matrices wheren is the number of potential
categories. The initial values of C and age are set to zero:C = 0,

age = 0.

(13)

If the categories J and j are connected, CJ,j changes fromzero to one.
A. Learning Algorithm of C-FART
We explain the learning algorithm of C-FART in detail.
Learning Steps(C-FART1) An input vector I is inputted to the category
layer F2 from the input layer F1 .
(C-FART2) A winning category J is chosen according to
the step (FART2). Furthermore, a second-winning category
J2 , whose TJ2 is the second largest next to TJ , is found for
updating connections if J2 exists.
(C-FART3) As the step (FART3), the similarity of the input
I and the current winning category w J is measured by thevigilance criterion of Eqs. (6) and (7).
(C-FART4) If any J satisfies Eq. (6), w J is updated by
Eq. (8) and we perform (C-FART5). If all the categories
do not satisfy Eq. (6), we check whether I is fully contained by a category k (as Fig. 2). In other words, if{min (wki , 1 − wkm+i ) < ai < max (wki , 1 − wkm+i )} is2008 International Joint Conference on Neural Networks (IJCNN 2008)

3595

StartInput I

Create newcategory
wn+1 = I

Find winner J
and second-winner J2

No
Is I fully
contained by any
category k ?

Learning
Is there
any wJ satisfying
Eq. (6) ?

No

Yes

Yes
I is classified into kUpdate weight vector wJNo

J2 exist
Yes

No

Eq. (15)
is satisfied

UpdateConnections

Create a connection
between J and J2
C J , J 2 = 1, ageJ , J 2 = 0

oldageJnew,i = ageJ , j + 1,

j ∈ NJ

Yes

ageJ , j > AT (t )CJ , j = 0

No
Is J overlapped withother category k ?

No

Yes

C J ,k = 1
YesNo

Combining
Categories

J and k are combined by Eq. (21)No

Finish learning
Yes
End

Fig. 1.

Flowchart of learning algorithm of C-FART.

true for all dimension i, I is classified to the category k, wk
is updated;
old
old
wnew
= β(I ∧ woldkk ) + (1 − β)w k ≡ w k ,(14)second-winning category wJ2 is measured by| I ∧ w J2 |
≥ ρ.
|I|

(15)

If Eq. (15) is satisfied, a connection between the winningcategory J and the second-winning category J2 is created;
CJ,J2 = 1.

and we perform (C-FART9) without the steps (C-FART5)(C-FART8). If more than one categories satisfy Eq. (6), thecategory k with the smallest index is chosen. If I is fully
contained by no categories, a new category is establishedaccording to Eq. (9) and we perform (C-FART9) without thesteps (C-FART5)-(C-FART8).

The age of the connection between J and J2 is set to zero(“refresh” the age);

Update Connections(C-FART5) If J2 does not exist, we skip this step andperform (C-FART6). The similarity of the input I and theOn the contrary, if Eq. (15) is not satisfied, the connection
is not updated(C-FART6) The age of all categories, which directly connect

3596

ageJ,J2 = 0.

2008 International Joint Conference on Neural Networks (IJCNN 2008)

(16)

(17)

with the winning category J, are increased one;old
agenew
J,j = ageJ,j + 1,

j ∈ NJ ,(18)

where NJ is the set of categories which directly connect withJ, namely CJ,j = 1.
(C-FART7) The connections are removed, if their age exceeds a threshold value AT (t);CJ,j = 0,
where

ageJ,j ≥ AT (t),AT (t) = ATi

ATf
ATi

t t

max

(19)

vJ

0.4

uk

0.3
0.1

,

wJ new = {(uJ ∧ v J ∧ uk ∧ uk ) , (ucJ ∧ v cJ ∧ uck ∧ uck )}.
(21)
The inputs belonging to the category k are classified tothe category J, and the category k is removed, therefore,
nnew = nold − 1.
(C-FART9) The steps from (C-FART1) to (C-FART8) arerepeated for all input data set. Therefore, C-FART makes
connections or releases connections at each step, and theoverlapped categories are combined with considering connections.
IV. S IMULATION R ESULTS
We apply the proposed C-FART to various input data andcompare C-FART with the conventional FART.
A. Simulation 1
First, in order to confirm the behavior of the proposed
C-FART, we consider simple 2-dimensional input data as
Fig. 4(a), consisting of 2-clusters, whose distribution is nonuniform. Total number of the input data is 500 points, andthe input data are sorted at random. 250 points are distributed
within a rectangular range from 0.0 to 0.2 horizontallyand from 0.0 to 0.6 vertically. The remaining 250 points
are distributed within a rectangular range from 0.4 to 0.6
horizontally and from 0.0 to 0.6 vertically. Both FART and
C-FART start learning with no categories, namely n = 0.
The parameters for the learning are chosen as follows;(For FART)α = 0.1, β = 1.0, ρ = 0.8,

uJnew

0.1

uJ
0.2

0.4 0.5

0.2

0.7

w J = (uJ , v Jc ) = (0.2, 0.1, 0.5, 0.6)

(20)

Combining Categories(C-FART8) We check whether the winning category
J is overlapped with other category and combine
these categories. We find the category k which areoverlapping with J as Figs. 3(a) and (c), in other words,{min (wJi , 1 − wJm+i ) < wki < max (wJi , 1 − wJm+i )}or {min (wJi , 1 − wJm+i )<1 − wkm+i<
max (wJi , 1 − wJm+i )} is true for any dimension i.
If k exists and directly connects with J, namely CJ,k = 1,
J and k are combined as Figs. 3(b) and (d) byv Jnew

0.6

w

w k = (uk , v kc ) = (0.7, 0.3, 0.6, 0.4)

where t is the learning step, tmax is the learning length,
ATi and ATf is the initial value and the final value of AT ,respectively.

(For C-FART)

vk
0.6

new
J

0.1

,v

)

(b)
v Jc

0.9

new

uk

0.3
0.2

= (u

c new
J

= (0.2, 0.1, 0.3, 0.4)

(a)
uJ

0.9

0.7
new
J

vJ

vk
0.2

uJnew

0.1
0.4

0.6

0.8

w J = (uJ , v Jc ) = (0.4, 0.9, 0.4, 0.8)
w k = (uk , v kc ) = (0.8, 0.3, 0.8, 0.9)

(c)

0.2

w

new
J

0.8

= (u

new
J

,v

c new
J

)

= (0.2, 0.1, 0.2, 0.1)

(d)Fig. 3. Overlapped categories combining process in C-FART. (a) Category
J and category k are overlapping. (b) If CJ,k = 1, w J is updated according
to Eq. 21, and k is removed. (c) Category J partially contains category k.
(d) If CJ,k = 1, w J and k are combined according to Eq. 21.

α = 0.1, β = 1.0, ρ = 0.8, ATi = 2, ATf = 24,
where we use the same α, β and ρ for the conventional
FART and the proposed C-FART for a fair comparison andthe confirmation of the combining effect.
Learning results of the conventional FART and the proposed C-FART are shown in Figs. 4(b) and (c), respectively.
We can see that the categories of C-FART are larger than
FART. This is because the proposed C-FART combines theoverlapping categories in consideration of their connections.
If all overlapping categories are combined, the input data
are not classified into each appropriate category since thehyperboxes keep getting larger and larger. However, the proposed C-FART makes connections between similar categories
or releases connections at each step, in addition, C-FART
combines the overlapping categories using the created connections. In other words, C-FART combines the categories
with due consideration of their similarity. In the result of
Fig. 4, the numbers of the categories of FART and CFART are 34 and 28, respectively. The proposed C-FART
reduces the category proliferation problem 20.00% from theconventional FART.
Furthermore, in order to appreciate the training performance of the two algorithms, we perform the learning
simulation 100 times to the input data created in the same
way as Fig. 4(a). The input data are created each time and2008 International Joint Conference on Neural Networks (IJCNN 2008)

3597

1

1

1

0.9

0.9

0.9

0.8

0.8

0.8

0.7

0.7

0.7

0.6

0.6

0.6

0.5

0.5

0.5

0.4

0.4

0.4

0.3

0.3

0.3

0.2

0.2

0.2

0.1

0.1

0
0

0.2

0.4

0.6

0.8

1

0
0

0.1
0.2

0.4

(a)

0.6

0.8

1

0
0

0.2

(b)

0.4

0.6

0.8

1

(c)

Fig. 4. Simulation of FART and C-FART for 2-clusters input data. (a) Input data. (b) Simulation result of conventional FART. (c) Simulation result of
proposed C-FART.

1

1

1

0.9

0.9

0.9

0.8

0.8

0.8

0.7

0.7

0.7

0.6

0.6

0.6

0.5

0.5

0.5

0.4

0.4

0.4

0.3

0.3

0.3

0.2

0.2

0.2

0.1

0.1

0
0

0.2

0.4

0.6

0.8

1

0
0

0.1
0.2

0.4

(a)

0.6

0.8

1

(b)

0
0

0.2

0.4

0.6

0.8

1

(c)

Fig. 5. Simulation of FART and C-FART for 3-clusters input data. (a) Input data. (b) Simulation result of conventional FART. (c) Simulation result of
proposed C-FART.

TABLE I

are sorted by random each time to remove the dependence
on the order of the input data.
Table I summarizes minimum, maximum and averagevalues of the number of categories. In other words, theminimum and the maximum values mean best and worst
results in 100 simulations, respectively. All the number ofcategories of the proposed C-FART are smaller than theconventional FART. From the average values, we can seethat C-FART can reduce the category proliferation problem
15.48% from using the conventional FART. We can say that
C-FART can obtain more effective result than FART.

data such as Fig. 5(a) into appropriate categories because thisdata contains clusters whose size are different. The learning
conditions are the same used in Simulation 1.
Figs. 5(b) and (c) show the learning results of the conventional FART and the proposed C-FART, respectively. We
can see that the conventional FART classifies data points
belonging to different clusters into same category. On theother side, the proposed C-FART can classify each input data
into each appropriate category although the hyperbox size of
C-FART is larger than FART.
Furthermore, we repeat this learning 100 times in the same
way as Simulation 1 and examine the minimum, maximum
and average values of the number of categories. The results
are summarized in Table II. We can see that all the numberof categories of the proposed C-FART are smaller thanthe conventional FART, and C-FART reduces categories byan average of 12.98%. This is because C-FART combines
the overlapping categories with due consideration of theirconnections.

B. Simulation 2

C. Simulation 3

Next, we apply C-FART to 1650 points input data whichhave 3-clusters as Fig. 5(a). The top-left cluster has 650
points, the bottom-left cluster has 200 points, and the bottomright cluster has 800 points. It is difficult to classify the input

Furthermore, we apply the proposed C-FART to rounded
input data as Fig. 6(a). The input data has two clusters:a round shape cluster and a ring-shaped cluster whose
distribution is non-uniform. The round shape cluster and the

N UMBER OF CATEGORIES IN S IMULATION 1.
Algorithm
FART
C-FART
Improvement rate

3598

Number of categories
Minimum
Maximum
Average
32
43
36.88
24
37
31.17
13.95%
25.00%
15.48%

2008 International Joint Conference on Neural Networks (IJCNN 2008)

1

1

1

0.9

0.9

0.9

0.8

0.8

0.8

0.7

0.7

0.7

0.6

0.6

0.6

0.5

0.5

0.5

0.4

0.4

0.4

0.3

0.3

0.3

0.2

0.2

0.2

0.1

0.1

0
0

0.2

0.4

0.6

0.8

1

0
0

0.1
0.2

0.4

(a)

(b)

0.6

0.8

1

0
0

0.2

0.4

0.6

0.8

1

(c)

Fig. 6. Simulation of FART and C-FART for circle input data. (a) Input data. (b) Simulation result of conventional FART. (c) Simulation result of proposed
C-FART.

TABLE IIN UMBER OF CATEGORIES IN S IMULATION 2.
Algorithm
FART
C-FART
Improvement rate

Number of categories
Minimum
Maximum
Average
57
70
62.77
48
63
54.62
10.00%
15.79%
12.98%TABLE IIIN UMBER OF CATEGORIES IN S IMULATION 3.
Algorithm
FART
C-FART
Improvement rate

Number of categories
Minimum
Maximum
Average
75
92
82.09
60
85
70.34
7.61%
20.00%
14.31%

In other words, C-FART combines the categories with due
consideration of their similarity. We have applied C-FART
to various input data set, and the learning behaviors of
C-FART have been investigated. Furthermore, in order toappreciate the training performance of the two algorithms,
we have performed the learning simulation 100 times. We
have confirmed that the proposed C-FART can obtain more
effective result than the conventional FART and can reducethe category proliferation problem.
ACKNOWLEDGMENTS
This work was partly supported by Yamaha Music Foundation.
R EFERENCES

ring-shaped cluster contain 300 and 700 points, respectively.
The learning conditions are the same as Simulation 1.
The learning result of the conventional FART and the proposed C-FART are shown in Fig. 6(b) and (c), respectively.
On the whole, the category size of C-FART is larger than
FART because C-FART combines the overlapping categories.
We repeat this learning 100 times and examine the minimum, maximum and average values of the number ofcategories. The input data are sorted by random each time.
The results are summarized in Table III. All the numberof categories of the proposed C-FART are smaller than theconventional FART, and C-FART reduces categories by anaverage of 14.31%. From these results, we can say that the
proposed C-FART can perform more effective learning thanthe conventional FART and can reduce the category proliferation problem by combining the overlapping categories withdue consideration of their connections.

[1] C.A. Carpenter, “Distributed learning, recognition, and prediction by
ART and ARTMAP neural networks,” Neural Networks, vol. 10, pp.
1473–1494, 1997.
[2] G.A. Carpenter, S. Grossberg, D.B. Rosen, “Fuzzy ART: Fast stablelearning and categorization of analog patterns by an adaptive resonancesystem,” Neural Networks, vol. 4, pp. 759–771, 1991.
[3] G.A. Carpenter, S. Grossberg, and D.B. Rosen, N. Markuzon,
J.H. Reynolds, D. B. Rosen, “Fuzzy ARTMAP: A neural network architecture for incremental supervised learning of Analog Multidimensional
Maps,” IEEE Trans. Neural Networks, vol. 3, no. 5, pp. 698–713, 1991.
[4] T. Frank, K.F. Kraiss and T. Kuhlen, “Competitive analysis of Fuzzy
ART and ART-2A network clustering performance,” IEEE Trans. Neural
Networks, vol. 9, no. 3, pp. 544–559, 1998.
[5] H. Isawa, M. Tomita, H. Matsushita and Y. Nishio, “Fuzzy Adaptive
Resonance Theory with Group Learning and its Applications,” Proc.
of International Symposium on Nonlinear Theory and its Applications,pp. 292–295, 2007
[6] A.V. Nandedkar, and P.K. Biswas, “Fuzzy Min-Max neural networkclassifier with compensatory neuron architecture,” IEEE Trans. Neural
Networks, vol. 18, no. 1, pp. 42–54, 2007
[7] L. Zadeh, “Fuzzy sets,” Information and Control, vol. 8, pp. 338–353,
1965.

V. C ONCLUSIONSIn this study, we have proposed Fuzzy ART Combining
Overlapped Category in consideration of connections (CFART) to solve the category proliferation problem. C-FART
makes connections between similar categories or releases
connections at each step, in addition, C-FART combines
the overlapping categories using the created connections.

2008 International Joint Conference on Neural Networks (IJCNN 2008)

3599