# Feature Selection in Decision Systems: A Mean-Variance Approach

Authors: Chengdong Yang; Wenyin Zhang; Jilin Zou; Shunbo Hu; Jianlong Qiu
Journal: Mathematical Problems in Engineering
 (2013)

---

## Abstract

Uncertainty measure is an important implement for characterizing the degree
of uncertainty. It has been extensively applied in pattern recognition and
data clustering. Because of instability of traditional uncertainty measures,
mean-variance measure (MVM) is utilized to perform feature selection, which
could depress disturbances and noises effectively. Thereby, a novel evaluation
function based on MVM is designed. The forward greedy search algorithm
(FGSA) with the proposed evaluation function is exploited to perform feature
selection. Experiment analysis shows the validity and effectiveness of MVM.


---

## Body

## 1. Introduction

Rough sets, originated by Pawlak [1] in 1980s, is a powerful mathematical tool to deal with inexact, uncertain, and vague knowledge in information systems. It has been drawing extensive attention in theory and applications in artificial intelligence, pattern recognition, data mining, intelligence information processing, decision support, image processing, feature selection, neural computing, conflict analysis, and knowledge discovery [2–10].Uncertainty measure is an important implement for characterizing the degree of uncertainty in rough set theory. It has been extensively applied in pattern recognition and data clustering. However, this paper reveals the issue that classical uncertainty measures are sensitive to disturbances or noises. Therefore, a novel uncertainty measure, called mean-variance measure (MVM), was proposed to characterize the degree of uncertainty of rough sets in paper [11]. Since it takes fully information in the boundary region into account, MVM is more robust and effective than classical uncertainty measures in depressing disturbances and noises.As an important application of rough sets in artificial intelligence and machine learning, feature selection or attribute reduction in information systems has been drawing wide attention. due to the fact that excessive features or attributes usually confuse learning algorithms, cause significant slowdowns in learning processes, and increase risks of learned classifiers to over-fit the training data [4, 12].Unfortunately, it has been proved that finding all reducts or finding an optimal reduct (a reduct with the least number of attributes) is an NP-complete problem [13]. Many researchers devote themselves to finding an efficient reduct by optimization techniques. The forward greedy search algorithm (FGSA), also called hill-climbing algorithm or greedy algorithm, is such an optimal technique for finding one reduct quickly and has been extensively investigated [14–16]. A key ingredient of FGSA lies in establishing an evaluation function to examine importance of each feature or attribute in databases. The evaluation function induced by the classical uncertainty measure, that is, the Pawlak’s roughness or dependency, has been successfully applied in rough sets based feature selection [17, 18]. Along with the development of rough sets, attribute reduction has been studied extensively in the past decade, such as fuzzy rough sets based attribute reduction [19–23], neighborhood rough sets based attribute reduction [24], cross-entropy based attribute reduction [25], tolerance rough sets based attribute reduction [26], cost based attribute reduction [27, 28], and dynamic attribute reduction [29, 30], and extended rough set based attribute reduction [31], cover rough sets based attribute reduction [14, 32], covering generalized rough sets based attribute reduction [33], variable precision rough sets based attribute reduction [34]. Nevertheless, the classical uncertainty measure is not robust and maybe fluctuates largely only with minor disturbances. Even a little change in information systems may produce an unpredictable fluctuation of this uncertainty measure.The mean value and variance in probability theory, able to be used to analyze preciously data, have been widely discussed in portfolio optimization and portfolio selection [35–38]. They are considered as an arbitrator which is used to determine whether a group of data is robust and stable. For example, two shooters obtain the same score (mean value). If one has to be chosen to take part in a tournament, which one should be chosen reasonably? Apparently, the one with a less variance score would like to be chosen. In this paper, the notions of mean value and variance are introduced into information systems as an arbitrator to evaluate the uncertainty degree. A novel uncertainty measure, called mean-variance measure (MVM), is proposed. MVM firstly calculates the mean of every object, and then all objects’ variances are taken into account. The effect caused by disturbances of data in decision systems on MVM will decrease, since a tiny alteration of values will not result in a large change of variance.Based on the new notion of MVM, an evaluation function called D-MVM in decision systems is further designed. The designed evaluation function takes full information in positive region and boundary region into account.This paper is organized as follows. Some elementary concepts on rough sets and MVM are reviewed in Section2.  Section 3 investigates the issue on feature selection in decision systems by MVM. Experimental results and analysis are given in Section 4, and Section 5 concludes this paper.


## 2. Preliminaries

### 2.1. Rough Sets

This section briefly outlines some basic notions on rough sets.Definition 1.
An information system is a pairS=(U,AT) satisfying (1)
Uis a nonempty finite set of objects;(2)
AT is a nonempty finite set of attributes;(3)
for everya∈AT, there is a mapping a:U→Va, where Va is the set of values.Definition 2.
Given an information systemS=(U,AT) and P⊆AT, an indiscernibility relation on U is defined by
(1)RP={(x,y)∈U×U∣a(x)=a(y),∀a∈P}.Obviously,RP is an equivalent relation induced by the attribute set P. [x]RP={y∈U∣(x,y)∈RP} is referred to as the equivalence class of x with respect to RP. A partition of U induced by the equivalent relation RP can be denoted by
(2)URP={P1,P2,…,Pn},
where Pi is some equivalence class of RP in U, i=1,2,…,n. U/RP and [x]RP are, respectively, denoted by U/P and [x]P, for short, when no ambiguity arises in this paper.Definition 3.
Given an information systemS=(U,AT), P⊆AT, and X⊆U, the lower approximation and the upper approximation of X with respect to P are defined, respectively, by
(3)P_(X)=⋃‍{Pi∈UP∣Pi⊆X},P¯(X)=⋃‍{Pi∈UP∣Pi∩X≠∅}.Definition 4.
Given an information systemS=(U,AT), a partial ordering relation ⪯ in the family {U/B∣B⊆AT} is defined as
(4)UP⪯UQ,
if and only if for any Pi∈U/P, there exists a Qj∈U/Q such that Pi⊆Qj, where U/P={P1,P2,…,Pn} and U/Q={Q1,Q2,…,Qm} are partitions induced by P,Q⊆A, respectively.
Q is said to be coarser than P, or P is finer than Q, ifU/P⪯U/Q. P is said to be strictly finer than Q, denoted by U/P≺U/Q, if U/P⪯U/Q but U/P≠U/Q.Proposition 5.
Given an information systemS=(U,AT), P,Q⊆AT, and if P⊆Q, then U/Q⪯U/P.From Proposition5, the more attributes an information system contains, the finer the corresponding partition is. Therefore, U/AT is the finest one among partitions induced by all subsets of AT.The classical uncertainty measure is defined as follows.Definition 6.
Given an information systemS=(U,AT) or an incomplete information system IS=(U,AT), P⊆AT, and X⊆U, the roughness of X is defined as
(5)ρP(X)=1-|P_(X)||P¯(X)|.The quantityρP(X) characterizes the uncertainty degree of X with respect to P. When ρP(X)=0, X is said to be definable; otherwise, it is said to be rough.WhenAT is divided into two nonempty sets C and D such that C∩D=∅, then S=(U,AT), denoted by S=(U,C∪D), is called a decision system, C is called the conditional attribute set, and D is called the decision attribute set.Definition 7.
Given a decision systemS=(U,C∪D),  the dependency degree of D on C is defined by(6)γ(C,D)=|POSC(D)||U|,
where POSC(D)=∪X∈U/DC_(X) is the positive region of D with respect to C and |*| denotes the cardinality of *.Definition 8.
Given a decision systemS=(U,C∪D) and B⊆C, B is independent if
(7)γ(B-a,D)<γ(B,D),∀a∈B.Attribute reduction in decision systems is defined as follows.Definition 9.
Given a decision systemS=(U,C∪D) and B⊆C, B is called a reduct if (1)
γ(B,D)=γ(C,D),(2)
γ(B-{a},D)<γ(B,D) for any a∈B.


### 2.2.  A Novel Uncertainty Measure of Rough Sets

Given an information systemS=(U,AT) and X⊆U, the characteristic function of X on U can be denoted by
(8)A(x)={1,ifx∈X,0,ifx∉X,
where x∈U.LetA(X)={A(x)/x∣x∈U}, then A(X) can be considered as a special fuzzy set derived from X on U.In rough set theory, objects in the same equivalent class cannot be distinguished for each other, since they have the same characteristic. However, in the boundary region of a rough set, objects in the same class have different characteristics. In this case, their mean value of objects in a class is generally used to characterize each object.Definition 10.
Given an information systemS=(U,AT), X⊆U, P⊆AT, and x∈U, the mean value of x in X, denoted by AP(x), is defined by
(9)AP(x)=|[x]P∩X||[x]P|.We denote{AR(x)/x∣x∈U} by AP(X). It is evident that AP(X) is a fuzzy set on U. As an example, given U={x1,x2,…,x36}, U/P={{x1,x2,x3,x4},{x5,x6,x7,x8},…,{x33,x34,x35,x36}}, and X={u8,…,x11,u14,…,x17,u20,…,x24,u26,…,x29}, seen in Figure 1(a), then A(x) and AP(x) are calculated by (8) and (9), respectively, as shown in Figures 1(b) and 1(c).X
,
A
(
X
), andAR(X).
(a)
X
(b)
A
(
X
)
(c)
A
R
(
X
)As mentioned above, when an objectx is not in X, its mean value is non-zero if and only if its equivalent class has non-empty intersection with X; when x is in X, its  mean value is 1 if and only if its its equivalent class is contained in X.  From Definition 10, it is easy to verify that the mean value AP(x) is an inclusion degree D(X/[x]P) of [x]P being included in X.Proposition 11.
Given an information systemS=(U,AT), X⊆U, P⊆AT, and x∈U, the following conclusions hold:(1)
if   x∈P_(X), then  AP(x)=1;(2)
ifx∉P¯(X), then  AP(x)=0;(3)
if   x∈P¯(X)-P_(X), then   0<AP(x)<1.Note thatAP(x)=A(x) when x is in the positive region and the negative region. It is obvious that AP(x)≠A(x) only when x is in the boundary region.Definition 12.
Given an information systemS=(U,AT), P⊆AT, and X⊆U, the mean-variance uncertainty measure (MVM) of X with respective to P, denoted by σP(X), is defined as
(10)σP(X)=∑x∈U(A(x)-AP(x))2|U|.It is clear that(11)σP(X)=∑x∈X(1-AP(x))2+∑x∈U-X(AP(x))2|U|.AssumeσP(X)=0 when X=∅ or P=∅. From Definition 12 one can see that only objects in the boundary region of X contribute to the value of σP(X). In this sense, σP(X) takes fully information in the boundary region into account. Therefore, it is a proper measure to evaluate the uncertainty of X.Definition 13.
Given an information systemS=(U,AT), P,Q⊆AT, and X,Y⊆U, (1)
X is said to be σ-definable if σP(X)=0;(2)
X is said to be σ-rough if σP(X)≠0;(3)
X is said to be coarser with respect to P than Y with respect to Q if σP(X)<σQ(Y), in which case, Y is called finer with respect to Q than X with respect to P.Next, we investigate properties ofσP(X) and show its efficiencies in evaluating uncertainty of a set in information systems.Proposition 14.
Given an information systemS=(U,AT), P⊆AT, and X⊆U, the following conclusions hold:(1)
σP(X)=0⇔ρP(X)=0;(2)
if   U/P=ω, then  σP(X)=0, where  ω={{x}∣x∈U}   is the finest partition of  U;(3)
if  U/P=δ, then   σP(X)=|X|/|U|-(|X|/|U|)2, where  δ={U}  is the coarsest partition of   U.


## 2.1. Rough Sets

This section briefly outlines some basic notions on rough sets.Definition 1.
An information system is a pairS=(U,AT) satisfying (1)
Uis a nonempty finite set of objects;(2)
AT is a nonempty finite set of attributes;(3)
for everya∈AT, there is a mapping a:U→Va, where Va is the set of values.Definition 2.
Given an information systemS=(U,AT) and P⊆AT, an indiscernibility relation on U is defined by
(1)RP={(x,y)∈U×U∣a(x)=a(y),∀a∈P}.Obviously,RP is an equivalent relation induced by the attribute set P. [x]RP={y∈U∣(x,y)∈RP} is referred to as the equivalence class of x with respect to RP. A partition of U induced by the equivalent relation RP can be denoted by
(2)URP={P1,P2,…,Pn},
where Pi is some equivalence class of RP in U, i=1,2,…,n. U/RP and [x]RP are, respectively, denoted by U/P and [x]P, for short, when no ambiguity arises in this paper.Definition 3.
Given an information systemS=(U,AT), P⊆AT, and X⊆U, the lower approximation and the upper approximation of X with respect to P are defined, respectively, by
(3)P_(X)=⋃‍{Pi∈UP∣Pi⊆X},P¯(X)=⋃‍{Pi∈UP∣Pi∩X≠∅}.Definition 4.
Given an information systemS=(U,AT), a partial ordering relation ⪯ in the family {U/B∣B⊆AT} is defined as
(4)UP⪯UQ,
if and only if for any Pi∈U/P, there exists a Qj∈U/Q such that Pi⊆Qj, where U/P={P1,P2,…,Pn} and U/Q={Q1,Q2,…,Qm} are partitions induced by P,Q⊆A, respectively.
Q is said to be coarser than P, or P is finer than Q, ifU/P⪯U/Q. P is said to be strictly finer than Q, denoted by U/P≺U/Q, if U/P⪯U/Q but U/P≠U/Q.Proposition 5.
Given an information systemS=(U,AT), P,Q⊆AT, and if P⊆Q, then U/Q⪯U/P.From Proposition5, the more attributes an information system contains, the finer the corresponding partition is. Therefore, U/AT is the finest one among partitions induced by all subsets of AT.The classical uncertainty measure is defined as follows.Definition 6.
Given an information systemS=(U,AT) or an incomplete information system IS=(U,AT), P⊆AT, and X⊆U, the roughness of X is defined as
(5)ρP(X)=1-|P_(X)||P¯(X)|.The quantityρP(X) characterizes the uncertainty degree of X with respect to P. When ρP(X)=0, X is said to be definable; otherwise, it is said to be rough.WhenAT is divided into two nonempty sets C and D such that C∩D=∅, then S=(U,AT), denoted by S=(U,C∪D), is called a decision system, C is called the conditional attribute set, and D is called the decision attribute set.Definition 7.
Given a decision systemS=(U,C∪D),  the dependency degree of D on C is defined by(6)γ(C,D)=|POSC(D)||U|,
where POSC(D)=∪X∈U/DC_(X) is the positive region of D with respect to C and |*| denotes the cardinality of *.Definition 8.
Given a decision systemS=(U,C∪D) and B⊆C, B is independent if
(7)γ(B-a,D)<γ(B,D),∀a∈B.Attribute reduction in decision systems is defined as follows.Definition 9.
Given a decision systemS=(U,C∪D) and B⊆C, B is called a reduct if (1)
γ(B,D)=γ(C,D),(2)
γ(B-{a},D)<γ(B,D) for any a∈B.


## 2.2.  A Novel Uncertainty Measure of Rough Sets

Given an information systemS=(U,AT) and X⊆U, the characteristic function of X on U can be denoted by
(8)A(x)={1,ifx∈X,0,ifx∉X,
where x∈U.LetA(X)={A(x)/x∣x∈U}, then A(X) can be considered as a special fuzzy set derived from X on U.In rough set theory, objects in the same equivalent class cannot be distinguished for each other, since they have the same characteristic. However, in the boundary region of a rough set, objects in the same class have different characteristics. In this case, their mean value of objects in a class is generally used to characterize each object.Definition 10.
Given an information systemS=(U,AT), X⊆U, P⊆AT, and x∈U, the mean value of x in X, denoted by AP(x), is defined by
(9)AP(x)=|[x]P∩X||[x]P|.We denote{AR(x)/x∣x∈U} by AP(X). It is evident that AP(X) is a fuzzy set on U. As an example, given U={x1,x2,…,x36}, U/P={{x1,x2,x3,x4},{x5,x6,x7,x8},…,{x33,x34,x35,x36}}, and X={u8,…,x11,u14,…,x17,u20,…,x24,u26,…,x29}, seen in Figure 1(a), then A(x) and AP(x) are calculated by (8) and (9), respectively, as shown in Figures 1(b) and 1(c).X
,
A
(
X
), andAR(X).
(a)
X
(b)
A
(
X
)
(c)
A
R
(
X
)As mentioned above, when an objectx is not in X, its mean value is non-zero if and only if its equivalent class has non-empty intersection with X; when x is in X, its  mean value is 1 if and only if its its equivalent class is contained in X.  From Definition 10, it is easy to verify that the mean value AP(x) is an inclusion degree D(X/[x]P) of [x]P being included in X.Proposition 11.
Given an information systemS=(U,AT), X⊆U, P⊆AT, and x∈U, the following conclusions hold:(1)
if   x∈P_(X), then  AP(x)=1;(2)
ifx∉P¯(X), then  AP(x)=0;(3)
if   x∈P¯(X)-P_(X), then   0<AP(x)<1.Note thatAP(x)=A(x) when x is in the positive region and the negative region. It is obvious that AP(x)≠A(x) only when x is in the boundary region.Definition 12.
Given an information systemS=(U,AT), P⊆AT, and X⊆U, the mean-variance uncertainty measure (MVM) of X with respective to P, denoted by σP(X), is defined as
(10)σP(X)=∑x∈U(A(x)-AP(x))2|U|.It is clear that(11)σP(X)=∑x∈X(1-AP(x))2+∑x∈U-X(AP(x))2|U|.AssumeσP(X)=0 when X=∅ or P=∅. From Definition 12 one can see that only objects in the boundary region of X contribute to the value of σP(X). In this sense, σP(X) takes fully information in the boundary region into account. Therefore, it is a proper measure to evaluate the uncertainty of X.Definition 13.
Given an information systemS=(U,AT), P,Q⊆AT, and X,Y⊆U, (1)
X is said to be σ-definable if σP(X)=0;(2)
X is said to be σ-rough if σP(X)≠0;(3)
X is said to be coarser with respect to P than Y with respect to Q if σP(X)<σQ(Y), in which case, Y is called finer with respect to Q than X with respect to P.Next, we investigate properties ofσP(X) and show its efficiencies in evaluating uncertainty of a set in information systems.Proposition 14.
Given an information systemS=(U,AT), P⊆AT, and X⊆U, the following conclusions hold:(1)
σP(X)=0⇔ρP(X)=0;(2)
if   U/P=ω, then  σP(X)=0, where  ω={{x}∣x∈U}   is the finest partition of  U;(3)
if  U/P=δ, then   σP(X)=|X|/|U|-(|X|/|U|)2, where  δ={U}  is the coarsest partition of   U.


## 3. Feature Selection in Decision Systems

In this section, the proposed uncertainty measure is further investigated to perform feature selection in decision systems.Definition 15.
Given a decision systemS=(U,C∪D) and B⊆C, MVM of the decision attribute set D with respect to the conditional attribute subset B, called D-MVM, or an evaluation function, is defined by
(12)σ(B,D)=1N(σB(D1)+σB(D2)+⋯+σB(DN)),
where N is the number of the decision classes induced by the decision attribute set D, σB(Di), i=1,2,…,N, reflect the uncertainty measure of each decision class, and σ(B,D) describes the integrated uncertainty degree of blocks D1,D2,…,DN.In the following, some properties ofσ(B,D) are studied.Proposition 16.
Given a decision systemS=(U,C∪D) and B⊆C, the following conclusions hold:(1)
σ(B,D)=0⇔ρ(B,D)=0;(2)
if   U/B=ω, then  σ(B,D)=0;(3)
if   U/B=δ, then   σ(B,D)=(1/N)∑i=1N|Di|/|U|-(|Di|/|U|)2.Proof.
The proof is analogous to that of ([11], property 2.14).Definition 17.
Given a decision systemS=(U,C∪D) and B⊆C, B is independent if
(13)σ(B,D)<σ(B-a,D),∀a∈B.By D-MVM, a relative reduct can be defined as follows.Definition 18.
Given a decision systemS=(U,C∪D) and B⊆C, B is a relative reduct of C with respective to D if and only if(1)
σ(B,D)=σ(C,D),(2)
σ(B,D)<σ(B-a,D) for any a∈B.A relative reduct is a minimal subset which has the same discriminating power as the raw decision systems.Definition 19 (significance  based  on  D-MVM).
Given a decision systemS=(U,C∪D), B⊆C, and a feature a∈C-B, the significance of a is defined as
(14)Sigσ(a,B,D)=σ(B∪a,D)-σ(B,D).Notice that ifB is an empty set, σ(B,D)=0, and Sigσ(a,B,D) is a nonnegative real number; otherwise, Sigσ(a,B,D)≤0.With the proposed evaluation function, a forward greedy search algorithm for feature selection can be designed as follows.In the first iteration, we start with an empty set specified withσ(B,D)=0. The quantity Sigσ(a,B,D) is negative in every iteration except the first one. The rest features in each iteration are all evaluated, and the one with the minimal significance will be chosen.  The algorithm does not stop until adding any of the rest features to selected feature set will not bring a change larger than threshold ɛ in Algorithm 1, where  ɛ controls the precision of the algorithm.Algorithm 1
Forward Greedy Search Algorithm of Feature Selection based on
Mean-Variable in Decision Systems (FGSA-MVM):
Input: (U, C∪D, V, f), ε
Output:red
(1)∅→red
(2) whileC−red≠∅
(3)  for eachai∈C−red
(4)   computeSigσ(ai, B, D) =σ(B∪ai,D)-σ(B,D)
(5)  end for
(6)  select the attributeak such that
(7)Sigσ(ak, red, D) = miniSigσ(ai, red, D)
(8)  if|Sigσ(ak,red,D)|<ε
(9)red∪ak→red
(10) else
(11)   break
(12) end if
(13) end while
(14) returnredThere is no doubt that FGSA-MVM is for the sake of searching a subset of conditional attributes with minimal positive real D-MVM. We obverse step 7 of FGSA-MVM. In the first iteration, we choose the minimal significance becauseSigσ(a,∅,D) is a positive number. In the rest iterations, we also select the minimal significance with the biggest step length since Sigσ(a,B,D) is nonpositive for any B≠∅.


## 4. Experiments and Analysis

In order to test the validity of the proposed method for feature selection, comparative experiments have been implemented in efficiency and convergence of proposed algorithm with two of the most important methods, feature selection based on dependence [39] and mutual information [40].As shown in Table1, four standard data sets, cited from the machine learning data repository, University of California,  Irvine, CA, USA [41], are employed in our experiments.Table 1
Data sets.
Data sets
Abbreviation
Samples
Features
Class
1
Lymphography
Lymph
148
19
4
2
Mushroom
Mush
8124
23
2
3
Soybean
Soybean
683
36
19
4
Zoo
Zoo
101
17
7CART and RBF-support vector machine (SVM) learning algorithms are introduced to test the classification performances of feature selection for raw sets and for selected feature sets. As a widely used technique to evaluate classification performances in machine learning, 10-fold cross-validation [42] is carried out in our experiments by dividing the samples into 10 subsets. Nine of them are used as training set, and the rest one is used as the test set. After 10 rounds, the average value and variation are computed as the final classification performance.Classification performances are evaluated by CART in Table2 and by RBF-SVM in Table 3. “Hold" marks the highest classification performances among these obtained by the methods based on three uncertainty measures. The number of selected features with the highest classification performance by the new measure is larger than that by dependency and by MI. It is 12, 4, and 12, respectively, via CART algorithm, whereas it is 14, 8, and 11 via SVM algorithm.Table 2
Comparison of classification performance of reducts based on different uncertainty measures with CART.
Data sets
Raw data
Dependency
MI
MVM
Lymph
0.6994
±
0.2195
0.6825
±
0.1822
0.6825
±
0.1822
0.6825
±
0.1822
Mush
0.9637
±
0.0990
0.9637
±
0.0990
0.9685
±
0.0996
0.9685
±
0.0996
Soybean
0.9174
±
0.0507
0.8485
±
0.0700
0.8780
±
0.0629
0.9192
±
0.0490
Zoo
0.9065
±
0.0913
0.8329
±
0.0676
0.9276
±
0.0987
0.9276
±
0.0987
Aver.
0.8567
0.8439
0.8672
0.8676Table 3
Comparison of classification performance of reducts based on different uncertainty measures with SVM.
Data sets
Raw data
Dependency
MI
MVM
Lymph
0.5623
±
0.0583
0.8448
±
0.0940
0.8448
±
0.0940
0.8448
±
0.0940
Mush
0.9587
±
0.0984
0.9587
±
0.0984
0.9587
±
0.0984
0.9587
±
0.0984
Soybean
0.5445
±
0.0649
0.6041
±
0.0723
0.6239
±
0.0683
0.6566
±
0.0817
Zoo
0.8615
±
0.0901
0.9239
±
0.0924
0.9239
±
0.0924
0.9239
±
0.0924
Aver.
0.7181
0.8435
0.8523
0.8549From the experiments one can see that the proposed measure outperforms not only in the smallest average number of selected features in reducts but also in the highest classification performance in feature selection.In the remainder of this section, we pay attention to the convergence of the proposed method. Figure2 shows the fluctuations of evaluation functions with respect to the number of selected features. The significance of selected features is calculated based on dependency, on MI, and on MVM, respectively. The four data sets are used to show the convergence of different techniques. The selected orders of the four data sets based on different evaluation functions are shown in Table 4, in which the sequences of selected features are different, even the number of selected features in the optimal reducts may be the same. As a whole, significance degrees based on dependency and MI increase, while significance based on MVM decreases. With MVM, all four evaluation functions decrease fast at the beginning of the selection process. The evaluation function of credit data slowly decreases, and this result constitutes a different pattern of behavior compared with the three other data sets. Feature selection algorithms may stop very early if we specify a threshold to stop the search in this case. The convergence and good classification performances are observed in the results.Table 4
Comparison of selected features by different uncertainty measures.
Data sets
Dependency
Mutual information
MVM
Lymph
18 2 13 14 15 16
13 18 14 2 15 16
13 18 14 2 15 16
Mush
5 20 8 12 3
5 20 22 21
5 20 22 21
Soybean
18 26 11 12 35 29 22 1
29 15 22 1 18 3 7 6
28 26 22 15 1 18 3 7
3 6 7 10 4 5 9 16
10 4 9 16 5
10 6 14 4
Zoo
4 13 12 6 8
13 4 6 8 3
13 4 6 8 3Comparison of convergence among three methods.
(a)
Dependency
(b)
MI
(c)
MVM


## 5. Conclusion

This contribution studied feature selection based on MVM in decision information systems, which is one of the most important applications of rough set theory. A novel approach to feature selection was proposed by introducing an evaluation function based on MVM. Theoretical analysis and experimental results concluded that the performances of proposed method are outperformed by dependency and by MI not only in the number of selected features but also in the classification precision.



---

*Source: 268063-2013-05-26.xml*