# A Simple Fitness Function for Minimum Attribute Reduction

Authors: Yuebin Su; Jin Guo; Zejun Li
Journal: Computational Intelligence and Neuroscience
 (2015)

---

## Abstract

The goal of minimal attribute reduction is to find the minimal subsetR of the condition attribute set C such that R has the same classification quality as C. This problem is well known to be NP-hard. When only one minimal attribute reduction is required, it was transformed into a nonlinearly constrained combinatorial optimization problem over a Boolean space and some heuristic search approaches were used. In this case, the fitness function is one of the keys of this problem. It required that the fitness function must satisfy the equivalence between the optimal solution and the minimal attribute reduction. Unfortunately, the existing fitness functions either do not meet the equivalence, or are too complicated. In this paper, a simple and better fitness function based on positive domain was given. Theoretical proof shows that the optimal solution is equivalent to minimal attribute reduction. Experimental results show that the proposed fitness function is better than the existing fitness function for each algorithm in test.


---

## Body

## 1. Introduction

For a given dataset, attribute reduction is a fundamental problem in rough set theory as proposed by Pawlak and Sowinski [1]. Formally, it is a nonlinearly constrained combinatorial optimization problem whose objective function is the size of a candidate subset of attributes and whose constraints, represented in terms of classification quality of attribute subsets, are the conditions met by an attribute reduction. As shown by [2], this problem has been proven to be NP-hard.To overcome this problem, many strategies had to be considered in the literatures during the past two decades. In general, there are two kinds of categories for minimum attribute reduction: greedy (or hill-climbing) categories and stochastic categories. The greedy categories usually employ rough set attribute significance as heuristic knowledge. It starts off with an empty set or attribute core and then adopts forward selection or backward elimination. Hu and Cercone give a reduction algorithm using the positive region-based attribute significance as the guiding heuristic [3]. Wang et al. develop a conditional information entropy-based reduction algorithm, using conditional entropy-based attribute significance [4]. Hu et al. compute the significance of an attribute making use of heuristic ideas from discernibility matrices and propose a heuristic reduction algorithm [5]. Susmaga considers both indiscernibility and discernibility relations in attribute reduction [6]. These categories are fast but do not guarantee to find an optimal or minimal reduction.Some researchers use stochastic methods for rough set attribute reduction. These categories are optimization methods. It has a higher probability of finding a minimum reduct than the first category. Wroblewski combines a genetic algorithm with a greedy algorithm to generate short reducts. However, it uses highly time-consuming operations and cannot assure that the resulting subset is really a reduct [7]. Taking Wroblewski’s work as a foundation, Bjorvand and Komorowski apply genetic algorithms to compute approximate reducts [8]. The algorithm makes several variations and practical improvements both in speed and in the quality of approximation. The reduct generation algorithms based on genetic algorithms for the rough set attribute reduction are quite efficient [9].But rough set can only deal with the discrete attributes. A method for discretization based on particle swarm optimization (PSO) is presented in [10]. Taking this work as a foundation, an algorithm for knowledge reduction in rough sets is proposed based on particle swarm optimization in [11]. This algorithm can solve some problems that the existing heuristic algorithm cannot solve. In order to improve the efficiency of the algorithm, many scholars constantly improve and update these algorithms. Santana-Quintero Luis et al. present a multiobjective evolutionary algorithm which consists of a hybrid between a particle swarm optimization approach and some concepts from rough sets theory [12]. The main idea of the approach is to combine the high convergence rate of the particle swarm optimization algorithm with a local search approach based on rough sets which is able to spread the nondominated solutions found. After that, two-step particle swarm optimization to solve the feature selection problem was given by Bello et al. in [13]. The improved algorithm is a method which can improve the search efficiency. Chi et al. presented a method for continuous attribute discretization based on quantum PSO algorithm [14]. Hsieh and Horng presented a method for feature selection based on asynchronous discrete PSO search algorithm [15]. Also, other stochastic algorithms were used to attribute reduction, for example, ant colony algorithm (ACO) [16] and support vector machine (SVR) [17].For systems where the optimal or minimal subset is required, stochastic category may be used. In this case, this problem is transformed into a problem of finding a maximum (or minimum) value of a fitness function at first, and then some stochastic optimization method is applied to solve the fitness maximization (or minimization) problem. A common way to transform a constrained optimization problem into an unconstrained fitness optimization problem is to use penalty methods [18]. For such methods, designing a better fitness function is the most important work. To get good performance, the fitness functions should meet the requirements that the fitness evaluation of a candidate solution is appropriate and the optimality equivalence is guaranteed. Here, the optimality equivalence means that the optimal solution of the fitness maximization problem corresponds to a minimum attribute reduction. Unfortunately, the existing fitness functions do not well meet the above mentioned requirements and consequently affect the performance of the related algorithms [19].In this paper, an applicable fitness function was proposed. Compared with the existing fitness functions as mentioned earlier, it not only takes into account less factors but also overcomes the drawback. The experimental results show that, for each of the two tested algorithms, the use of the proposed fitness function has a higher probability to find a minimum reduction than the use of the function proposed in [19].The rest of the paper is organized as follows. Section2 presents some concepts about minimum attribute reduction and reviewed and analysed a fitness function proposed in [19]. In Section 3, a new fitness function and properties are presented. In Section 4, the results of experiments and comparison analysis are given. Finally, Section 5 concludes the paper.


## 2. Basic Notions and Related Works

In this section, we will review some basic notions in the theory of rough sets which are necessary for the description of the minimum attribute reduction problem.A decision table can be represented asS
=
{
U
,
A
,
V
,
f
}, where U
=
{
x
1
,
x
2
,
…
,
x
n
} is a nonempty finite set of objects, A
=
C
∪
D, where C is a set of condition attributes and D is a decision attribute set, V is the domain of attributes belonging to A, and f
:
U
×
A
→
V is a function assigning attribute values to objects in U.With anyR
⊆
A, there is an associated indiscernibility relation I
N
D
(
R
):(1)
I
N
D
R
=
x
,
y
f
x
,
a
=
f
y
,
a
,
∀
a
∈
R
,
x
,
y
∈
U
.LetX
⊆
U; the R-lower approximation of X is defined as R
_
X
=
{
x
∈
U
∣
[
x
]
R
⊆
X
}, where [
x
]
R denotes an equivalence class of I
N
D
(
R
) determined by object x. The notation P
O
S
R
(
D
) refers to the R-positive region given by P
O
S
R
D
=
∪
X
∈
U
/
I
N
D
(
D
)
R
_
X. The R-approximation quality with respect to decisions attribute set D is defined as γ
R
=
|
P
O
S
R
(
D
)
|
/
|
U
|.For decision tableS
=
{
U
,
A
,
V
,
f
}, it may have many attribute reductions; the set of reductions is defined as(2)
R
e
d
=
R
⊆
C
∣
P
O
S
R
D
=
P
O
S
C
D
,
∀
B
⊂
R
,
P
O
S
R
D
≠
P
O
S
B
D
.For attribute reduction, the minimal attribute reduction with minimal cardinality will be searched. The minimal attribute reduction problem can be formulated as the following nonlinearly combinational optimization problem:(3)
m
i
n
R
⊂
C
R
s
.
t
.
P
O
S
R
D
=
P
O
S
C
D
,
P
O
S
R
D
≠
P
O
S
B
D
,
∀
B
⊂
R
.LetR
e
d
m be a set which was defined as follows:(4)
R
e
d
m
=
R
∈
R
e
d
∣
∀
B
∈
R
e
d
,
B
≥
R
.By the definition ofR
e
d
m, the following proposition is apparent.Proposition 1.
LetR
⊂
C; then R is the optimal solution of the optimization problem (3) if and only if R
∈
R
e
d
m.According to Proposition1, each element of R
e
d
m corresponds to a minimal reduction. In order to solve problem (3), the most commonly used approach is to transform it into the following unconstrained maximization problem and to solve it by heuristic algorithms:(5)
m
a
x
R
⊂
C
F
R
,where F
(
R
) is a fitness function about attribute subset R.For this optimization problem, the equivalence of optimality between the minimum attribute reduction problem (3) and the fitness maximization of the function F
(
R
) must be guaranteed. Unfortunately, most of the functions do not satisfy the requirement in the literatures [19].For instance, Ye et al. defined a fitness function in [19]. Let(6)
I
N
D
C
=
x
1
C
,
x
2
C
,
…
,
x
N
C
,where the cardinality of the equivalence classes [
x
i
]
C meets |
[
x
i
]
C
|
≤
|
[
x
i
+
1
]
C
|, where · denotes the cardinality of a set. The function was defined as(7)
F
1
R
=
m
-
R
m
+
n
Δ
γ
R
,where Δ
=
|
[
x
1
]
C
|
>
0.LetR
⊂
C, |
R
|
=
|
C
|
-
1, and |
P
O
S
R
(
D
)
|
=
|
P
O
S
C
(
D
)
|
-
1; then F
1
(
C
)
=
|
P
O
S
C
(
D
)
|
/
Δ and F
1
(
R
)
=
1
/
m
+
(
|
P
O
S
C
(
D
)
|
-
1
)
/
Δ, and thus F
1
(
C
)
-
F
1
(
R
)
=
1
/
Δ
-
1
/
m. If 1
/
Δ
=
1
/
m, then F
1
(
C
)
-
F
1
(
R
)
=
0. It suggests that the fitness function cannot distinguish between the two. That is to say, the two are equivalent when running the algorithm. This drawback will reduce the algorithmic performance. Moreover, in practical applications, since contains classification quality γ
R, the value of the function F
1
(
R
) is not an integer. In the search process, the calculation error will reduce the probability of searching a minimum reduction. At the end, the consistency of decision table must be determined before calculating the fitness value, which increases the computational complexity.


## 3. A New Function and Its Properties

In this section, we present a new fitness function. Firstly, the relevant definitions were introduced as follows.Definition 2.
LetR
⊆
C. If P
O
S
R
(
D
)
=
P
O
S
C
(
D
) holds, then R is said to be a consistent set of C. Let C
S be a set which includes all the consistent sets of C [19]:(8)
C
S
=
R
∣
P
O
S
R
D
=
P
O
S
C
D
,
R
⊆
C
.By the definition of reduction, for any consistent setR
∈
C
S, R either is a reduction itself or contains a reduction. If R is minimal, then R is a reduction of C. It is very obvious that R
e
d
⊆
C
S. A nonlinearly constrained combinatorial optimization problem correlated with C
S is defined as(9)
m
i
n
R
∈
C
S
R
.LetC
S
m be a set which is defined as follows:(10)
C
S
m
=
R
∈
C
S
:
∀
B
∈
C
S
,
B
≥
R
.By the definition ofC
S
m, we know that C
S
m is a subset of C
S and the following proposition is apparent.Proposition 3.
∀
R
∈
C
S
m, then R is a reduction.Proposition 4.
LetR
⊂
C; then R
∈
C
S
m if and only if R is the optimal solution of the optimization problem (9).By Proposition4, C
S
m is the solution set of the optimization problem (9). According to the previous definition, we present a new fitness function as follows: ∀
R
⊆
C,(11)
F
R
=
m
P
O
S
C
D
-
P
O
S
R
D
+
R
,where m
=
|
C
|.Proposition 5.
LetP
⊂
C.
(
1
) If P
∈
C
S
,
∀
R
∈
C
S
m
i
n, then F
(
R
)
≤
F
(
P
)
≤
m.
(
2
) If P
∉
C
S, then F
(
P
)
>
m.
In particular,F
(
P
)
>
F
(
B
)
,
∀
B
∈
C
S.Proof.
IfP
∈
C
S, ∀
R
∈
C
S
m
i
n, then P
O
S
P
D
=
P
O
S
R
D
=
P
O
S
C
D and |
P
|
≤
|
R
|
≤
m. That is F
(
R
)
≤
F
(
P
)
≤
m according to the definition of the fitness function F
(
R
). The (
1
) is hold. If P
∉
C
S, then |
P
O
S
C
(
D
)
|
>
|
P
O
S
P
(
D
)
|, thus F
(
P
)
=
m
(
|
P
O
S
C
(
D
)
|
-
|
P
O
S
P
(
D
)
|
)
+
|
P
|
>
m
+
|
P
|
>
m.According to Proposition5, the codomain of the function F
(
R
) can be divided into two disjoint sets [
m
i
n
F
(
R
)
,
m
] and (
m
,
m
a
x
F
(
R
)
), where m
i
n
F
(
R
) and m
a
x
F
(
R
) are the minimum and maximum value of F
(
R
), respectively. By (
2
), the defect of the function proposed in [19] was avoided. A minimum optimization problem is defined as follows by the definition of F
(
R
):(12)
m
i
n
R
⊂
C
F
R
.Theorem 6.
LetR
⊂
C; then R
∈
C
S
m if and only if R is a minimum attribute reduction; that is, C
S
m
=
R
e
d
m.Proof.
R
∈
C
S
m; then R is a reduction by Proposition 3. If R is not a minimum attribute reduction, then there is a minimum attribute reduction B such that |
B
|
<
|
R
|.
On the other hand, sinceB is an attribute reduction, then B
∈
C
S. By the definition of C
S
m and R
∈
C
S
m, |
B
|
≥
|
R
|, a contradiction and vice versa.By Theorem6, all the elements of C
S
m are minimum attribute reduction; then according to Proposition 4, we can get that all the solutions of the optimization problem (9) are the minimum attribute reductions.Theorem 7.
∀
R
⊂
C
S
m, then R is an optimal solution of the optimization problem (12). It is equivalent to the case where all the elements of C
S
m are the optimal solutions of the optimization problem (12).Proof.
We use proof by contradiction. Assume that there is aR
∈
C
S
m such that R is not an optimal solution of the optimization problem (12). Then ∃
B
⊆
C such that F
(
B
)
<
F
(
R
). So(13)
0
>
F
B
-
F
R
=
m
P
O
S
R
D
-
P
O
S
B
D
+
B
-
R
.Then, there are two cases of B that will be discussed in the following.
(
1
) If B
∈
C
S, then P
O
S
B
(
D
)
=
P
O
S
R
(
D
)
=
P
O
S
C
(
D
), so |
P
O
S
B
(
D
)
|
-
|
P
O
S
R
(
D
)
|
=
0 and F
(
B
)
-
F
(
R
)
=
|
R
|
-
|
B
|. Since R
∈
C
S
m, thus |
R
|
-
|
B
|
≤
0. Consequently (14)
0
>
F
B
-
F
R
=
B
-
R
≥
0
,a contradiction.
(
2
) If B
∉
C
S, then |
P
O
S
B
(
D
)
|
<
|
P
O
S
R
(
D
)
|
=
|
P
O
S
C
(
D
)
|; that is, |
P
O
S
R
(
D
)
|
-
|
P
O
S
B
(
D
)
|
≥
1, and then(15)
0
>
F
B
-
F
R
≥
m
+
B
-
R
≥
B
≥
1
,a contradiction.From the above proof we know that all the elements ofC
S
m are the optimal solution of the optimization problem (12); that is to say, the minimum attribute reduction is the optimal solution of the optimization problem (12).Theorem 8.
LetR
⊂
C; if R is an optimal solution of the optimization problem (12), then R
∈
C
S
m.Proof.
(
1
)  R
∈
C
S.
IfR
∉
C
S, then |
P
O
S
R
(
D
)
|
<
|
P
O
S
C
(
D
)
|. For all B
∈
C
S, P
O
S
B
(
D
)
=
P
O
S
C
(
D
). Since R is an optimal solution of the optimization problem (12), then 0
≥
F
(
R
)
-
F
(
B
), so(16)
0
≥
F
R
-
F
B
=
m
P
O
S
B
D
-
P
O
S
R
D
+
R
-
B
≥
m
+
R
-
B
≥
1
,a contradiction, so R
∈
C
S.
(
2
)  R
∈
C
S
m.
IfR
∉
C
S
m, let B
∈
C
S
m. According to (
1
) we know R
∈
C
S, so P
O
S
B
(
D
)
=
P
O
S
R
(
D
), and then (17)
0
≥
F
R
-
F
B
=
R
-
B
>
0
,a contradiction, so R
∈
C
S
m.According to Theorem8, the optimal solution of the optimization problem (12) is a minimum attribute reduction. Then, by Theorems 7 and 8, we can obtain the following theorem obviously.Theorem 9.
LetR
⊂
C; then R is an optimal solution of the optimization problem (12) if and only if R
∈
C
S
m.By Theorem9, we can get that the optimal solution of the optimization problem (12) is equivalent to the minimal attribute reduction. Then, according to Theorem 6, we can obtain the following theorem.Theorem 10.
LetR
⊂
C; then R is a minimum attribute reduction if and only if R is an optimal solution of the optimization problem (12).By Theorem10, the minimum value of the function proposed in this paper is equivalent to the minimal attribute reduction. We may get the minimum attribute reduction by searching the minimum value of function, and then stochastic category can be used.


## 4. Performance Comparison

In order to analyze and evaluate the effectiveness of the fitness function proposed in this paper, in the literature [19], an experiment was designed in the following way. Two existing minimum attribute reduction algorithms based on different types of stochastic optimization technique are used in the comparison. The first is the particle swarm optimization-based attribute reduction algorithm proposed in [20], denoted by PSO, and the other is a genetic algorithm based attribute reduction algorithm presented in [7, 8, 16], denoted by GA in short. In experiments, both algorithms were implemented and tested on a number of datasets using two different fitness functions: one is proposed in [19] (F
1 in short) and the new fitness function is proposed in this paper (F in short). Seven datasets were chosen from the UCI machine learning repository. Most of these datasets are commonly used for evaluating attribute reduction algorithms in the literature [7, 8, 11, 14, 16, 17, 19].For each of the two fitness functions, both algorithms were run 50 times on each of the datasets with the same setting of the parameters involved. For each run, three values need to be recorded: the first is the length of the output, the second is the output which corresponds to a reduction or not, and the last is run time. If an output is a reduction, then the output is said to be a normal output; otherwise, it is an unsuccessful output. If the length of the normal output is minimal, then the output is said to be a successful output. Let STL denote the length of the successful output, AVL denote the average length, and AVT denote the average run time during the 50 runs. The ratios of successful and normal outputs are denoted, respectively, bys
1 and s
2.A PC running Windows 7 (32-bit) with 2.1 × 2 GHz CPU and 2 GB of main memory was used to run both algorithms. Both algorithms were programed by the MATLAB. Parameter settings for both algorithms were shown in Table1. In Table 1, p-size refers to the size of population (in GA) or particle swarm (in PSO), I
m
a
x is the maximum allowed number of iterations (or generations), V
m
a
x is the upper bound on velocity needed in the PSO algorithm, m
p and c
p are the probabilities of mutation and crossover in GA, and c
1 and c
2 are the learning coefficients in PSO. Tables 2 and 3 present the main performance of PSO and GA using each of the two fitness functions.Table 1
Parameter settings for both algorithms.
Algorithms
p-size
V
max
⁡
I
max
⁡
c
1
c
2
m
p
c
p
PSO
20
7
100
2.0
2.0
—
—
Gen
50
—
100
—
—
0.4
0.6Table 2
Performance of PSO using different fitness functions.
Dataset
F
1
F
STL
AVL
s
1
/
s
2
AVT
STL
AVL
s
1
/
s
2
AVT
blance
4
4
1/1
35.392
4
4
1/1
26.467
spect
16
16
1/1
7.471
16
16
1/1
6.921
Lung-cancer
3
4.02
0.08/1
2.628
3
3.92
0.22/1
1.096
Sponge
8
8.4
0.61/0.98
5.327
8
8.02
0.79/1
4.217
Vote
8
8.2
0.8/1
7.419
8
8.13
0.87/1
5.522
Zoo
5
5.05
0.96/1
5.092
5
5
1/1
4.275
Soybean-small
2
2.32
0.78/1
2.091
2
2.38
0.72/1
1.162Table 3
Performance of GA using different fitness functions.
Dataset
F
1
F
STL
AVL
s
1
/
s
2
AVT
STL
AVL
s
1
/
s
2
AVT
blance
4
4
1/1
38.501
4
4
1/1
30.359
spect
16
16
1/1
8.021
16
16
1/1
7.856
Lung-cancer
3
4.21
0.07/0.98
2.983
3
4.03
0.2/1
1.123
Sponge
8
8.58
0.59/0.96
6.306
8
8.17
0.77/0.98
4.311
Vote
8
8.31
0.78/1
8.762
8
8.22
0.82/1
5.762
Zoo
5
5.11
0.94/1
6.243
5
5
1/1
4.52
Soybean-small
2
2.43
0.76/1
2.227
2
2.51
0.71/1
1.175From Table2, for the algorithm PSO, on the index of STL, both fitness functions have the same value. It means that both fitness functions can output the minimum reduction. On the index of AVL, the value of F is not more than F
1’s except for the last date. It reflects that the output of F is more focused on STL. The same conclusion can be arrived at from the index s
1. It means that it has a higher probability to find a minimum reduction by using the proposed fitness function. For AVT, all the values of F are less than the values of F
1. Therefore, the experimental data show that the efficiency of the proposed fitness function is better than F
1 by using the algorithm PSO. The same conclusion can be obtained from Table 3 by using the algorithm GA.The above two experiments show that the proposed fitness function is more adequate than the other fitness functions on the datasets.


## 5. Conclusions

In this paper, in order to overcome the drawback of the existing fitness functions for the problem of minimum attribute reduction, we discussed the fitness function and a simpler fitness function was proposed in this paper. Theoretical analysis and experimental results show that it can ensure the optimality equivalence and is more adequate than the existing fitness function.



---

*Source: 921487-2015-08-03.xml*