# A Novel Robust Fuzzy Rough Set Model for Feature Selection

Authors: Yuwen Li; Shoushui Wei; Xing Liu; Zhimin Zhang
Journal: Complexity
 (2021)

---

## Abstract

The existing fuzzy rough set (FRS) models all believe that the decision attribute divides the sample set into several “clear” decision classes, and this data processing method makes the model sensitive to noise information when conducting feature selection. To solve this problem, this paper proposes a robust fuzzy rough set model (RS-FRS) based on representative samples. Firstly, the fuzzy membership degree of the samples is defined to reflect its fuzziness and uncertainty, and RS-FRS model is constructed to reduce the influence of the noise samples. RS-FRS model does not need to set parameters for the model in advance and can effectively reduce the complexity of the model and human intervention. On this basis, the related properties of RS-FRS model are studied, and the sample pair selection algorithm (SPS) based on RS-FRS is used for feature selection. In this paper, RS-FRS is tested and analysed on the open 12 datasets. The experimental results show that RS-FRS model proposed can effectively select the most relevant features and has certain robustness to the noise information. The proposed model has a good applicability for data processing and can effectively improve the performance of feature selection.


---

## Body

## 1. Introduction

In the current era of big data, the data scale is massive, and the presentation is high-dimensional. The high dimension of data representation is mainly due to that data often contains a large number of redundant or irrelevant features, resulting in excessively high data dimension, which seriously reduces the processing capacity and time efficiency of pattern classification as well as the resolution ability of decision making. High-dimensional data also makes the fast, timely, and accurate data mining task face great challenges. Therefore, how to effectively select features for these data has become one of the hot topics in the field of machine learning [1, 2]. The purpose of feature selection is to remove a large number of irrelevant and redundant features from the original feature set on the premise of ensuring the learning performance to find a set of feature subsets containing all or most of the classification information of the original feature space to reduce the impact of “dimension disaster” and improve the learning performance. Therefore, feature selection (or attribute reduction) is very necessary, which has become a research hotspot in machine learning. Meanwhile, fuzzy rough set (FRS) theory is not only an objective and effective mathematical tool to deal with incomplete and uncertain information [3–7] but also a powerful and effective computing paradigm to realize feature selection [8–10]. In recent years, FRS theory has been widely concerned and applied in data mining, machine learning, pattern recognition, and other fields [11–13].FRS can effectively deal with the fuzziness and vagueness of data. However, based on the upper and lower approximation of the classical FRS model, the nearest sample of the given target sample is used in the calculation, so the classical FRS model constrained by the nearest sample is extremely sensitive to noise.In order to improve the robustness of classical FRS and reduce the influence of noise samples on the approximation under the model, many robust FRS models were proposed [14–20]. An important application of FRS theory is feature selection, also known as attribute reduction, which maintains the inconsistency between feature and decision label. This inconsistency is manifested as two samples having the same feature value but different decision labels. Redundant or irrelevant features can be deleted to improve the classification prediction performance of learning algorithm and save running time and space, so that people can have a clearer understanding of the actual problems based on FRS. Feature selection based on FRS refers to the removal of redundant and irrelevant features from data without changing the ability of data classification. The advantage of FRS method is that in the process of data processing, only the information of the data itself can be utilized without any other prior knowledge and additional information. Generally speaking, there are two types of FRS-based feature selection methods: heuristic method based on dependency [21–23] and a structured method based on a discernibility matrix [24–28]. The dependency-based heuristic method uses the positive region and the dependency function [21–23] as the feature evaluation criterion and uses the heuristic search to obtain the feature subset. Specifically, FRS-based feature selection is calculated by using the forward or backward searching method on the premise of keeping a certain metric constant. Pawlak [1] introduced the concept of positive region when describing the consistency between features and decision label to select features. Hu and Cercone [23] designed an algorithm for calculation of reduction under the condition that the positive region of the decision was kept unchanged. Another kind of method is structured method based on discernible matrix. By introducing the identification matrix, a Boolean identification function is constructed, and the minimal form of the identification function is obtained by logical operation, thus all possible reduction results of the decision table are obtained. Based on the identification information in the identification matrix, many scholars have studied the computational problems of reduction [24–28]. For example, Yao and Zhao [24] transformed an identification matrix into its minimal form by designing an absorption matrix algorithm to calculate the reduction, so that the union of all elements constituted a reduction. Because this algorithm needs to calculate all the elements in the identifiable matrix, it will consume a lot of calculation time. In order to save a lot of running time and storage space, Chen et al. [27, 28] determined minimal elements of discernibility matrix through sample pair selection method (SPS) and designed a fast algorithm for calculation reduction based on minimal elements.In our previous work [29], we proposed a novel FRS model, namely, fuzzy rough set, with representative sample (RS-FRS) model, which is constructed to reduce the influence of noisy samples. In the RS-FRS model, the fuzziness of sample membership is taken into consideration. Using fuzzy equivalent approximate space, other subsets of domain space can be approximated more precisely than those from a conventional FRS model. RS-FRS model does not require preset parameters. Our pilot study indicates that implementing such a new framework could reduce the complexity of the model and human intervention.However, the previous work needs further research in several aspects. Firstly, the proposed theorem which supports the model is not thoroughly derived, which calls for additional theoretical derivation to strengthen the mathematical background. Moreover, the verification of the proposed model is not comprehensive since the model was tested using only the KNN classifier. Lastly, the previous work lacks solid statistical validations due to its nature as a pilot study that explores the method’s potential effectiveness.In this manuscript, we extensively addressed these drawback points. We conducted detailed derivations of Properties2 and 3 to complete the RS-FRS model at the theoretical level. The completion of the theorem also supports the generation of the feature selection algorithm, which is significant to the framework based on RS-FRS models. In addition to the previous KNN classifier, the performance evaluation of the proposed method is extended by validating the algorithm using CART and LSVM classifiers. Our new results show satisfactory accuracy and robustness values, which further support this new method’s effectiveness. Lastly, a comprehensive statistical analysis of the results is conducted and reported in this manuscript by applying Friedman tests and Bonferroni-Dunn tests to the model outputs. In summary, the previous pilot study is strengthened with theoretical, experimental, and statistical proofs to demonstrate the performance and robustness of the proposed RS-FRS method.


## 2. Preliminaries

The equivalence classes of Pawlak rough set are crisp subsets of the domain. These crisp information granules cannot reflect the fuzziness in reasoning. In practical classification learning, the features describing samples may be fuzzy, or the relations between samples are fuzzy relations calculated by numerical attributes. Therefore, FRS as Pawlak rough set extension model came into being.For a nonempty universeU, if R is a binary relation and it satisfies reflexivity, symmetry, and sup-min transitivity, R is a fuzzy equivalence relation. The fuzzy equivalence class xR is generated by R with respect to sample x∈U. xR is a fuzzy set on U, which is also referred as the fuzzy neighborhood of x, i.e., xRu=Rx,u, for all u∈U.Definition 1.
Given a nonempty finite domainU=x1,x2,…,xn, R is the fuzzy equivalence relation of U. For ∀x∈U, the fuzzy equivalence class of x is xiR=ri1/x1+ri2/x2+⋯+rin/xn, and it is the fuzzy subset of U. The membership degree value of ∀xj∈U to xiR is xiRxj=rij. The set of fuzzy equivalence classes forms a basic conceptual system for approximating any subset in the theoretic domain space, which is called fuzzy equivalence approximate space FAS=U,R. The upper and lower approximations of X∈FU in U,R are defined as(1)R¯Xx=infy∈Umax1−Rx,y,Xy,R¯Xx=supy∈UminRx,y,Xy.
In equation (1), the upper approximation of the sample x∈U is determined by Rx,y and Xy. The lower approximation of the sample x∈U is determined by 1−Rx,y and Xy, where Xy is the membership degree of the sample y∈U with respect to the equivalence class X∈FU.
At present, the existing fuzzy rough set models [14–20] consider that the decision attribute D divides the sample set into several crisp decision classes, so the membership degree Xy in the upper and lower approximations of the FRS model is a binary function with the value of 0 or 1. Then, the upper and lower approximations of the classical FRS model are reduced to(2)R¯Dix=infu∉Di1−Rx,u,R¯Dix=supu∈DiRx,u.


## 3. The Proposed Model

At present, the existing FRS models [14–20] consider that the decision attribute D divides the sample set U into several crisp decision classes, so Xy as the membership degree in equation (1) is a binary function with the value of 0 or 1. Then, the upper and lower approximations of classical FRS model are degenerated as equation (2). However, this strategy that it divides the sample set into crisp decision classes is extremely sensitive to the noise samples, and if there exist noise samples, Xy has fuzzy uncertainty. Only defining Xy as a binary function of 0 or 1 does not meet the requirements of practical application, and it cannot well reflect the real membership relations between samples and each equivalence class. Therefore, determining the membership of samples is another important challenge for fuzzy rough set models.


### 3.1. Representative Sample

Unlike the existing FRS models, we considerD divides U into several fuzzy decision classes and define “representative sample” to calculate the fuzzy membership of samples. To be specific, the corresponding representative samples are found for each label. Then, the fuzzy membership degree of the target sample with respect to each label is calculated according to the distance between the target sample and the representative sample, so as to design a robust FRS model.Definition 2.
(see [29]). Let U,A∪D be a fuzzy decision system, where the sample set U=x1,x2,…,xn has m attributes A=a1,a2,…,am. The decision attribute D divides the sample set U into r crisp equivalent decision classes U/D=D1,D2,…,Di,…,Dr1≤i≤r. The representative sample RSi of the class Di is defined as(3)RSi=argx∈Dimin∑y∈Didx,y,where dx,y is the distance between two samples in the class Di. In this paper, Euclidean distance is used as a basic implementation of dx,y.Definition 3.
(see [29]). Let U,A∪D be a fuzzy decision system, where the sample set U=x1,x2,…,xn has m attributes A=a1,a2,…,am. The decision attribute D divides the sample set U into r crisp equivalent decision classes U/D=D1,D2,…,Di,…,Dr1≤i≤r. The representative sample of the class Di is RSi. The membership degree of the sample x∈U with respect to class Di is defined as(4)Dix=dx,RSi∑k=1rdx,RSk,where dx,RSi is the distance between sample x and representative sample RSi.
According to Definition3, we can determine the membership degree of sample x with respect to each equivalence class by calculating the distance between sample x and the representative samples of each equivalence class. The membership degree Dix meets Dix∈0,1 and ∑i=1rDix=1. It can be seen that Dix can fully reflect the fuzziness of sample membership. The larger the value of Dix, the higher the degree of sample x belongs to class Di. The smaller the value of Dix, the lower the degree of sample x belongs to class Di. In the dataset, the samples located in the boundary region may be the noise samples that have been mislabeled. By the calculation of fuzzy membership, the degree of the boundary samples belonging to each class can be determined.


### 3.2. FRS Model with Representative Sample

Based on Definitions2 and 3, we propose a FRS model with representative samples (RS-FRS).Definition 4.
(see [29]). Let U,A∪D be a fuzzy decision system, where the sample set U=x1,x2,…,xn has m attributes A=a1,a2,…,am. The decision attribute D divides the sample set U into r crisp equivalent decision classes U/D=D1,D2,…,Di,…,Dr1≤i≤r. R is a fuzzy equivalence relation on U. The representative sample of the class Di is RSi. The upper and lower approximations of RS-FRS model are defined as(5)RRS¯Dix=infu∈Umax1−Rx,u,Diu,RRS¯Dix=supu∈UminRx,u,Diu,where Diu=du,RSi/∑k=1rdu,RSk.
The lower approximation of the FRS model indicates the certainty that a sample belongs to its decision class in the fuzzy approximation space, and the upper approximation indicates the possibility that a sample belongs to its decision class in the fuzzy approximation space. Therefore, the lower approximation can be used for classification and feature selection.
The samples of dataset located in the classification boundary region are most likely to be noise samples. Because these samples are close to the samples of other classes, these noise samples are often used to calculate the lower approximation of the classical FRS model. This process causes the lower approximation to get smaller. In the proposed RS-FRS model, we consider not only the distance between the target sample and the nearest different classes’ sample but also the fuzzy membership of the nearest different classes’ sample. The fuzzy membership degree is calculated to expand the lower approximation of the model and reduce the influence of noise sample on the lower approximation, thus RS-FRS is robust.
The main difference between the RS-FRS model and the classical FRS is that the fuzziness of sample membership is ignored when the upper and lower approximations are calculated based on the classical FRS. This can easily lead to errors in the upper and lower approximations of classical FRS model when dealing with datasets containing noise samples. Therefore, the classical FRS can only maintain the maximum fuzzy dependence, but it cannot process noise information well. In contrast, the RS-FRS model considers the fuzziness of sample membership degree and can more precisely approximate other subsets of the domain space with fuzzy equivalent approximation space, so that the data fitting effect is better. Compared with the existing robust FRS model, the RS-FRS model does not need to set parameters for the model in advance, which can effectively reduce the model complexity and human intervention.


### 3.3. Related Properties of FRS Model with Representative Sample

For standard max operatorSx,y=maxx,y, standard min operator Tx,y=minx,y, and standard complement operator Nx=1−x, some properties of RS-FRS model are discussed. If other fuzzy operators are used [13], the relevant conclusions can be similarly generalized.Property 1.
(see[29]). For ∀A∈FU, the following statements hold:(6)1RRS¯A=NRRS¯A,2RRS¯A=NRRS¯A.Proof.
∀x∈U(7)NRRS¯Ax=Nsupu∈UminRx,u,Au=infu∈UNminRx,u,Au=infu∈Umax1−Rx,u,Au=RRS¯A,NRRS¯Ax=Ninfu∈Umax1−Rx,u,Au=supu∈UNmax1−Rx,u,Au=supu∈UminRx,u,Au=RRS¯A.Property 2.
For∀A∈FU, if x is a normal sample, the following statements hold:(8)RRS¯A⊆A⊆RRS¯A.Proof.
∀x∈U(9)RRS¯Ax=infu∈Umax1−Rx,u,Au≤max1−Rx,u,Au.
Becausex is a normal sample, x can be used directly to compute its lower approximation RRS¯Ax,(10)max1−Rx,u,Au=max1−Rx,x,Ax=max0,Ax=Ax.
Therefore,RRS¯A⊆A.(11)RRS¯Ax=supu∈UminRx,u,Au≥minRx,u,Au.
Becausex is a normal sample, x can be used directly to compute its upper approximation RRS¯Ax,(12)minRx,u,Diu=minRx,x,Ax=min1,Ax=Ax.
Therefore,A⊆RRS¯A.Property 3.
For∀A∈FU, RT¯A and RS¯A are the upper and lower approximations of the classical FRS, respectively. The following statements hold:(13)1RRS¯A⊇RS¯A,2RRS¯A⊆RT¯A.Proof.
∀x∈U, according to equation (2) in Section 2, when the standard operator is given, then(14)RRS¯Ax=infu∈Umax1−Rx,u,Au≥inf1−Rx,u=RS¯A.
Therefore,RRS¯A⊇RS¯A.(15)RRS¯Ax=supu∈UminRx,u,Au≤supRx,u=RT¯A.
Therefore,RRS¯A⊆RT¯A.
Property1 proves the duality between the upper and lower approximations. Property 2 demonstrates the inclusion relationship between the upper and lower approximations and the original sample set. Property 3 proves the relationship of the upper and lower approximations between the RS-FRS model and the classical FRS model. The presentation and derivation of these properties provide a theoretical basis for the establishment and design of subsequent algorithms.


## 3.1. Representative Sample

Unlike the existing FRS models, we considerD divides U into several fuzzy decision classes and define “representative sample” to calculate the fuzzy membership of samples. To be specific, the corresponding representative samples are found for each label. Then, the fuzzy membership degree of the target sample with respect to each label is calculated according to the distance between the target sample and the representative sample, so as to design a robust FRS model.Definition 2.
(see [29]). Let U,A∪D be a fuzzy decision system, where the sample set U=x1,x2,…,xn has m attributes A=a1,a2,…,am. The decision attribute D divides the sample set U into r crisp equivalent decision classes U/D=D1,D2,…,Di,…,Dr1≤i≤r. The representative sample RSi of the class Di is defined as(3)RSi=argx∈Dimin∑y∈Didx,y,where dx,y is the distance between two samples in the class Di. In this paper, Euclidean distance is used as a basic implementation of dx,y.Definition 3.
(see [29]). Let U,A∪D be a fuzzy decision system, where the sample set U=x1,x2,…,xn has m attributes A=a1,a2,…,am. The decision attribute D divides the sample set U into r crisp equivalent decision classes U/D=D1,D2,…,Di,…,Dr1≤i≤r. The representative sample of the class Di is RSi. The membership degree of the sample x∈U with respect to class Di is defined as(4)Dix=dx,RSi∑k=1rdx,RSk,where dx,RSi is the distance between sample x and representative sample RSi.
According to Definition3, we can determine the membership degree of sample x with respect to each equivalence class by calculating the distance between sample x and the representative samples of each equivalence class. The membership degree Dix meets Dix∈0,1 and ∑i=1rDix=1. It can be seen that Dix can fully reflect the fuzziness of sample membership. The larger the value of Dix, the higher the degree of sample x belongs to class Di. The smaller the value of Dix, the lower the degree of sample x belongs to class Di. In the dataset, the samples located in the boundary region may be the noise samples that have been mislabeled. By the calculation of fuzzy membership, the degree of the boundary samples belonging to each class can be determined.


## 3.2. FRS Model with Representative Sample

Based on Definitions2 and 3, we propose a FRS model with representative samples (RS-FRS).Definition 4.
(see [29]). Let U,A∪D be a fuzzy decision system, where the sample set U=x1,x2,…,xn has m attributes A=a1,a2,…,am. The decision attribute D divides the sample set U into r crisp equivalent decision classes U/D=D1,D2,…,Di,…,Dr1≤i≤r. R is a fuzzy equivalence relation on U. The representative sample of the class Di is RSi. The upper and lower approximations of RS-FRS model are defined as(5)RRS¯Dix=infu∈Umax1−Rx,u,Diu,RRS¯Dix=supu∈UminRx,u,Diu,where Diu=du,RSi/∑k=1rdu,RSk.
The lower approximation of the FRS model indicates the certainty that a sample belongs to its decision class in the fuzzy approximation space, and the upper approximation indicates the possibility that a sample belongs to its decision class in the fuzzy approximation space. Therefore, the lower approximation can be used for classification and feature selection.
The samples of dataset located in the classification boundary region are most likely to be noise samples. Because these samples are close to the samples of other classes, these noise samples are often used to calculate the lower approximation of the classical FRS model. This process causes the lower approximation to get smaller. In the proposed RS-FRS model, we consider not only the distance between the target sample and the nearest different classes’ sample but also the fuzzy membership of the nearest different classes’ sample. The fuzzy membership degree is calculated to expand the lower approximation of the model and reduce the influence of noise sample on the lower approximation, thus RS-FRS is robust.
The main difference between the RS-FRS model and the classical FRS is that the fuzziness of sample membership is ignored when the upper and lower approximations are calculated based on the classical FRS. This can easily lead to errors in the upper and lower approximations of classical FRS model when dealing with datasets containing noise samples. Therefore, the classical FRS can only maintain the maximum fuzzy dependence, but it cannot process noise information well. In contrast, the RS-FRS model considers the fuzziness of sample membership degree and can more precisely approximate other subsets of the domain space with fuzzy equivalent approximation space, so that the data fitting effect is better. Compared with the existing robust FRS model, the RS-FRS model does not need to set parameters for the model in advance, which can effectively reduce the model complexity and human intervention.


## 3.3. Related Properties of FRS Model with Representative Sample

For standard max operatorSx,y=maxx,y, standard min operator Tx,y=minx,y, and standard complement operator Nx=1−x, some properties of RS-FRS model are discussed. If other fuzzy operators are used [13], the relevant conclusions can be similarly generalized.Property 1.
(see[29]). For ∀A∈FU, the following statements hold:(6)1RRS¯A=NRRS¯A,2RRS¯A=NRRS¯A.Proof.
∀x∈U(7)NRRS¯Ax=Nsupu∈UminRx,u,Au=infu∈UNminRx,u,Au=infu∈Umax1−Rx,u,Au=RRS¯A,NRRS¯Ax=Ninfu∈Umax1−Rx,u,Au=supu∈UNmax1−Rx,u,Au=supu∈UminRx,u,Au=RRS¯A.Property 2.
For∀A∈FU, if x is a normal sample, the following statements hold:(8)RRS¯A⊆A⊆RRS¯A.Proof.
∀x∈U(9)RRS¯Ax=infu∈Umax1−Rx,u,Au≤max1−Rx,u,Au.
Becausex is a normal sample, x can be used directly to compute its lower approximation RRS¯Ax,(10)max1−Rx,u,Au=max1−Rx,x,Ax=max0,Ax=Ax.
Therefore,RRS¯A⊆A.(11)RRS¯Ax=supu∈UminRx,u,Au≥minRx,u,Au.
Becausex is a normal sample, x can be used directly to compute its upper approximation RRS¯Ax,(12)minRx,u,Diu=minRx,x,Ax=min1,Ax=Ax.
Therefore,A⊆RRS¯A.Property 3.
For∀A∈FU, RT¯A and RS¯A are the upper and lower approximations of the classical FRS, respectively. The following statements hold:(13)1RRS¯A⊇RS¯A,2RRS¯A⊆RT¯A.Proof.
∀x∈U, according to equation (2) in Section 2, when the standard operator is given, then(14)RRS¯Ax=infu∈Umax1−Rx,u,Au≥inf1−Rx,u=RS¯A.
Therefore,RRS¯A⊇RS¯A.(15)RRS¯Ax=supu∈UminRx,u,Au≤supRx,u=RT¯A.
Therefore,RRS¯A⊆RT¯A.
Property1 proves the duality between the upper and lower approximations. Property 2 demonstrates the inclusion relationship between the upper and lower approximations and the original sample set. Property 3 proves the relationship of the upper and lower approximations between the RS-FRS model and the classical FRS model. The presentation and derivation of these properties provide a theoretical basis for the establishment and design of subsequent algorithms.


## 4. Feature Selection Based on FRS Model with Representative Sample

Although the method based on discernibility matrix [27] can be used for feature selection, it has two obvious disadvantages: (1) a waste of computing time cost. All feature subsets can be obtained by using discernibility matrix method. However, when the result of feature selection is used for pattern recognition or classification, only one feature subset is sufficient. In other words, feature selection based on discernibility matrix method will increase invalid time cost. (2) A waste of storage space. When using the discernibility matrix, it is necessary to store all the elements in this matrix and reduce them through absorption operator. However, the key to find one feature subset is to determine the minimum elements. Compared with storing the minimum elements, the discernibility matrix method will cause a waste of storage space. Therefore, in this paper, we use the improved discernibility matrix method, named sample pair selection algorithm (SPS) [28]. Since each of these minimum elements is determined by at least one sample pair, SPS algorithm can select the pairs of samples corresponding to the minimum elements in the sample set. Based on this, we propose a feature selection algorithm based on the RS-FRS model with SPS to save time and space costs.LetU,A∪D be a fuzzy decision system, where the sample set U=x1,x2,…,xn has m attributes A=a1,a2,…,am. Each conditional attribute at∈A induces a fuzzy relation Rat. Attribute subset B⊆A induces the fuzzy relation RB=∩at∈BRat. The discernibility matrix of this decision system is a n×n matrix, denoted by MDU,A, and the element of discernibility matrix is cij:(16)1cij=at:1−Ratxi,xj≥λi,λi=RA¯xiDxi,ifxj∉xiD;2cij=∅,otherwise,where λi is the lower approximation of xi with respect to xiD.In the RS-FRS model, the relative discrimination relation of conditional attributeat with respect to decision attribute D is defined as a binary relation, which is calculated by the following equation:(17)DIS′Rat=xi,xj∈U×U|1−Ratxi,xj≥λi,xi∉xiD,where λi=RA¯SRxiDxi and DIS′RA=∪at∈ADIS′Rat.We obtain the following results by finding the relationship betweenDIS′Rat and element cij in the discernible matrix MDU,A. If cij≠∅, xi,xj∈DIS′Rat⟺at∈cij. Nij=at|xi,xj∈DIS′Rat, where Nij is the number of conditional attributes that satisfied xi,xj∈DIS′Rat. Obviously, Nij=cij.According to the above definitions and analysis, a feature selection algorithm based on RS-FRS model with SPS can be described by Algorithm1.Algorithm 1:Feature selection algorithm based on RS-FRS model with SPS.
Input: a set of condition attributes A; a set of samples U.Output: selected feature subset S(1)
∀xi∈U, according to equation (5), the lower approximation λi of RS-FRS model is computed;(2)
Compute everyDIS′Rat and DIS′RA;(3)
Sortxi,xj∈DIS′Rat according to Nij;(4)
whileDIS′RA≠∅do(5)
Selected the first sample pairxi0,xj0∈DIS′RA;(6)
Select oneat such that xi,xj∈DIS′Rat and add at into S;(7)
DIS′RA=DIS′RA−DIS′Rat;(8)
end while(9)
returnS


## 5. Experiments

In this section, we compare the proposed RS-FRS model with several existing models from the three aspects of classification accuracy, robustness, and statistical properties. The existing models include FRS [30], β-PFRS (β=0.5) [14], K-trimmed FRS (k = 5) [18], K-means FRS (k = 5) [18], K-median FRS (k = 5) [18], and SFRS (r = 0.1) [15]. To test the performance of all FRS models in feature selection, these experiments are conducted on three common classifiers, including k-nearest neighbor (KNN, K = 3) [31], classification regression tree (CART) [32], and linear support vector machine (LSVM) [33].


### 5.1. Datasets

In our experiments, 12 datasets from the open access UCI database [34] are utilized and described in Table 1. UCI dataset is a commonly used dataset of machine learning standard tests. For example, in dataset “wine”, these data are the results of a chemical analysis of wines grown in the same region in Italy but derived from three different cultivars. The analysis determined the quantities of 13 constituents found in each of the three types of wines. The attributes are alcohol, malic acid, ash, alkalinity of ash, magnesium, total phenols, flavanoids, nonflavanoid phenols, proanthocyanins, color intensity, hue, OD280/OD315 of diluted wines, and proline.Table 1
Descriptions of 12 datasets.
No.DatasetsSamplesAttributesClasses1Glass214962Wine1781333Heart2701324Segment23101875Hepatitis1551926ICU2002037German10002028Soy472149Horse36822210WDBC56930211WPBC19833212Sonar208602


### 5.2. Results

#### 5.2.1. Comparison of Classification Accuracy

On three classifiers, we conduct comparison of classification accuracy with different FRS models in original data and noise data. The corresponding noise levels are 0%, 5%, and 10%, respectively. 0% noise level means the original data (the original data itself assumes no noise).5% (or 10%) noise level means there are 5% (or 10%) samples mislabeled randomly in the original data. In order to ensure the reliability of the experiments, we carry out the 10-fold cross-validation based on each independent noise random process. Tables2–4 show the comparison of classification performance of different FRS models on KNN, CART, and LSVM in original and noisy data. Bold values indicate the optimal performance among all FRS models, and italic values indicate the average classification accuracy of each model under all datasets.Table 2
Comparison of classification performance of different FRS models on KNN in original and noisy data.
DatasetsNoise level (%)FRSβ-PFRSK-trimmed FRSK-means FRSK-median FRSSFRSRS-FRSGlass00.678 ± 0.1340.650 ± 0.0680.631 ± 0.0610.650 ± 0.0780.655 ± 0.0990.678 ± 0.1340.668 ± 0.03150.629 ± 0.0610.649 ± 0.0270.630 ± 0.0370.647 ± 0.0910.648 ± 0.0430.655 ± 0.0960.656 ± 0.029100.628 ± 0.0980.647 ± 0.0660.626 ± 0.1100.645 ± 0.0190.644 ± 0.0680.648 ± 0.0890.649 ± 0.015Wine00.931 ± 0.0720.926 ± 0.0540.926 ± 0.0540.931 ± 0.0720.931 ± 0.0720.931 ± 0.0720.931 ± 0.07250.892 ± 0.0690.910 ± 0.0450.925 ± 0.0590.930 ± 0.0590.924 ± 0.0620.920 ± 0.0730.928 ± 0.086100.876 ± 0.0580.909 ± 0.0380.924 ± 0.0350.912 ± 0.0720.908 ± 0.0540.919 ± 0.0710.926 ± 0.044Heart00.744 ± 0.0690.759 ± 0.0820.741 ± 0.1050.730 ± 0.0780.730 ± 0.0780.744 ± 0.0690.744 ± 0.06950.735 ± 0.0710.739 ± 0.0590.740 ± 0.1100.728 ± 0.0800.727 ± 0.0130.741 ± 0.0670.742 ± 0.038100.719 ± 0.0820.733 ± 0.0860.734 ± 0.0440.726 ± 0.0190.727 ± 0.0780.736 ± 0.0540.736 ± 0.025Segment00.946 ± 0.1090.949 ± 0.1090.953 ± 0.1090.955 ± 0.1090.953 ± 0.1090.946 ± 0.0330.946 ± 0.10950.944 ± 0.0380.948 ± 0.0360.949 ± 0.0320.951 ± 0.0920.952 ± 0.0340.945 ± 0.0550.945 ± 0.010100.941 ± 0.0400.947 ± 0.0320.946 ± 0.0530.949 ± 0.0390.947 ± 0.0370.944 ± 0.0710.944 ± 0.104Hepatitis00.765 ± 0.1070.765 ± 0.1070.765 ± 0.1070.765 ± 0.1070.765 ± 0.1070.765 ± 0.1070.765 ± 0.10750.763 ± 0.0920.764 ± 0.1080.763 ± 0.0450.764 ± 0.0990.760 ± 0.1230.764 ± 0.0920.764 ± 0.062100.750 ± 0.1180.759 ± 0.0750.760 ± 0.0750.764 ± 0.0810.753 ± 0.0770.750 ± 0.0280.762 ± 0.104ICU00.868 ± 0.1620.879 ± 0.1830.879 ± 0.1830.868 ± 0.1290.868 ± 0.1290.868 ± 0.1620.880 ± 0.04250.856 ± 0.1760.859 ± 0.0870.860 ± 0.1580.857 ± 0.1620.855 ± 0.1460.861 ± 0.0540.863 ± 0.107100.850 ± 0.0230.855 ± 0.1260.853 ± 0.1010.856 ± 0.0230.852 ± 0.1760.856 ± 0.0100.862 ± 0.093German00.716 ± 0.0370.712 ± 0.0610.713 ± 0.0450.713 ± 0.0320.713 ± 0.0320.716 ± 0.0370.716 ± 0.03750.708 ± 0.0380.710 ± 0.0460.710 ± 0.0590.709 ± 0.0370.712 ± 0.0410.713 ± 0.0630.714 ± 0.031100.706 ± 0.0330.710 ± 0.0350.708 ± 0.0420.708 ± 0.0910.709 ± 0.0320.710 ± 0.0250.713 ± 0.088Soy00.850 ± 0.1410.825 ± 0.1340.850 ± 0.1410.850 ± 0.1410.850 ± 0.1410.850 ± 0.1410.850 ± 0.14150.790 ± 0.0620.819 ± 0.0770.818 ± 0.0250.818 ± 0.0180.816 ± 0.0500.820 ± 0.0410.820 ± 0.112100.760 ± 0.0460.818 ± 0.0820.810 ± 0.0820.806 ± 0.0870.809 ± 0.2460.816 ± 0.0920.818 ± 0.070Horse00.864 ± 0.0560.845 ± 0.0510.834 ± 0.0600.834 ± 0.0600.840 ± 0.0610.845 ± 0.0510.864 ± 0.05650.830 ± 0.0420.829 ± 0.0610.823 ± 0.0720.830 ± 0.0550.825 ± 0.0730.828 ± 0.0520.828 ± 0.081100.815 ± 0.0560.825 ± 0.0480.818 ± 0.0650.829 ± 0.0610.824 ± 0.0740.826 ± 0.0140.827 ± 0.099WDBC00.937 ± 0.0360.955 ± 0.0190.956 ± 0.0230.946 ± 0.0340.952 ± 0.0340.937 ± 0.0280.937 ± 0.03650.928 ± 0.0320.933 ± 0.0410.951 ± 0.0200.941 ± 0.0370.952 ± 0.0290.936 ± 0.0710.936 ± 0.111100.920 ± 0.0270.927 ± 0.0210.949 ± 0.0380.940 ± 0.0340.951 ± 0.0670.935 ± 0.0830.935 ± 0.027WPBC00.757 ± 0.0500.727 ± 0.0830.721 ± 0.1010.711 ± 0.0950.721 ± 0.0820.726 ± 0.1010.757 ± 0.05050.721 ± 0.0870.721 ± 0.0660.712 ± 0.1140.711 ± 0.0980.711 ± 0.0690.722 ± 0.0810.751 ± 0.114100.720 ± 0.0890.720 ± 0.0530.710 ± 0.0950.709 ± 0.1080.706 ± 0.1010.719 ± 0.0770.747 ± 0.093Sonar00.731 ± 0.0670.774 ± 0.0600.779 ± 0.0830.769 ± 0.0980.769 ± 0.0980.731 ± 0.0670.780 ± 0.09150.730 ± 0.0630.750 ± 0.0550.763 ± 0.0640.765 ± 0.1060.764 ± 0.0770.729 ± 0.0130.772 ± 0.123100.728 ± 0.0670.744 ± 0.0890.726 ± 0.0700.726 ± 0.0750.730 ± 0.0590.728 ± 0.0880.761 ± 0.105Average0.798 ± 0.0720.805 ± 0.0690.804 ± 0.0740.804 ± 0.0740.804 ± 0.0800.804 ± 0.0680.812 ± 0.073Table 3
Comparison of classification performance of different FRS models on CART in original and noisy data.
DatasetsNoise level (%)FRSβ-PFRSK-trimmed FRSK-means FRSK-median FRSSFRSRS-FRSGlass00.478 ± 0.0310.436 ± 0.0270.454 ± 0.0390.458 ± 0.0450.455 ± 0.0250.478 ± 0.0310.478 ± 0.03150.430 ± 0.0250.434 ± 0.0260.433 ± 0.0100.436 ± 0.0040.434 ± 0.0490.463 ± 0.0690.465 ± 0.060100.429 ± 0.0690.431 ± 0.0140.432 ± 0.0250.431 ± 0.0510.430 ± 0.0520.454 ± 0.0100.460 ± 0.025Wine00.909 ± 0.0680.909 ± 0.0680.908 ± 0.0810.908 ± 0.0810.908 ± 0.0810.908 ± 0.0810.909 ± 0.06850.859 ± 0.0590.903 ± 0.0920.901 ± 0.0260.860 ± 0.0750.899 ± 0.0640.902 ± 0.0720.903 ± 0.083100.819 ± 0.0760.857 ± 0.0370.897 ± 0.0870.859 ± 0.0410.893 ± 0.0560.896 ± 0.0540.902 ± 0.107Heart00.707 ± 0.0190.759 ± 0.0990.752 ± 0.0890.707 ± 0.0190.707 ± 0.0190.707 ± 0.1240.752 ± 0.05250.694 ± 0.0540.737 ± 0.0910.707 ± 0.0190.702 ± 0.0470.704 ± 0.0100.705 ± 0.0550.750 ± 0.049100.659 ± 0.0450.728 ± 0.0430.693 ± 0.0110.682 ± 0.0440.693 ± 0.0310.704 ± 0.0230.748 ± 0.076Segment00.947 ± 0.0290.946 ± 0.0390.945 ± 0.0350.945 ± 0.0350.945 ± 0.0350.947 ± 0.0290.947 ± 0.02950.941 ± 0.0310.953 ± 0.0270.942 ± 0.0410.943 ± 0.0340.942 ± 0.0270.945 ± 0.0830.946 ± 0.082100.938 ± 0.0360.938 ± 0.0300.940 ± 0.0320.941 ± 0.0350.941 ± 0.0380.942 ± 0.0640.944 ± 0.033Hepatitis00.800 ± 0.0630.800 ± 0.0630.800 ± 0.0630.800 ± 0.0630.800 ± 0.0630.800 ± 0.0630.800 ± 0.06350.762 ± 0.0930.761 ± 0.1030.770 ± 0.1300.753 ± 0.0810.755 ± 0.1010.796 ± 0.0390.798 ± 0.080100.723 ± 0.1260.752 ± 0.0790.732 ± 0.0210.751 ± 0.1040.728 ± 0.1200.781 ± 0.0410.790 ± 0.105ICU00.799 ± 0.0850.783 ± 0.0360.783 ± 0.0360.783 ± 0.0360.783 ± 0.0360.799 ± 0.0850.799 ± 0.08550.760 ± 0.0620.764 ± 0.0240.763 ± 0.0220.773 ± 0.0450.761 ± 0.0190.762 ± 0.0770.786 ± 0.091100.751 ± 0.0130.755 ± 0.0880.754 ± 0.0960.755 ± 0.0190.754 ± 0.0490.752 ± 0.0820.761 ± 0.114German00.716 ± 0.0370.704 ± 0.0350.713 ± 0.0450.713 ± 0.0320.713 ± 0.0320.712 ± 0.0460.716 ± 0.03750.716 ± 0.0530.700 ± 0.0310.711 ± 0.0390.712 ± 0.0340.710 ± 0.0380.710 ± 0.0710.713 ± 0.064100.707 ± 0.0330.700 ± 0.0610.709 ± 0.0310.708 ± 0.0420.709 ± 0.0320.710 ± 0.0790.709 ± 0.088Soy00.915 ± 0.1110.915 ± 0.1110.915 ± 0.1110.915 ± 0.1110.915 ± 0.1110.915 ± 0.1110.915 ± 0.11150.810 ± 0.0670.883 ± 0.0770.879 ± 0.0040.891 ± 0.0670.878 ± 0.0580.894 ± 0.0680.893 ± 0.108100.790 ± 0.0390.810 ± 0.0510.806 ± 0.0580.808 ± 0.0130.806 ± 0.0250.821 ± 0.0220.825 ± 0.024Horse00.855 ± 0.0460.862 ± 0.0490.857 ± 0.0500.854 ± 0.0470.857 ± 0.0500.856 ± 0.0520.855 ± 0.04650.846 ± 0.0670.860 ± 0.0530.854 ± 0.0600.853 ± 0.0500.852 ± 0.0470.855 ± 0.0130.855 ± 0.103100.841 ± 0.0470.848 ± 0.0550.848 ± 0.0380.851 ± 0.0330.853 ± 0.0400.853 ± 0.0510.854 ± 0.027WDBC00.907 ± 0.0370.931 ± 0.0460.907 ± 0.0420.926 ± 0.0440.916 ± 0.0380.907 ± 0.0370.928 ± 0.02950.903 ± 0.0840.924 ± 0.0310.903 ± 0.0410.909 ± 0.0330.914 ± 0.0340.905 ± 0.1020.927 ± 0.057100.895 ± 0.0580.923 ± 0.0220.905 ± 0.0450.907 ± 0.0320.896 ± 0.0360.904 ± 0.0460.924 ± 0.037WPBC00.660 ± 0.1150.703 ± 0.1280.645 ± 0.1090.660 ± 0.1340.634 ± 0.1590.666 ± 0.0900.660 ± 0.09250.630 ± 0.1180.702 ± 0.1120.640 ± 0.1050.656 ± 0.1530.631 ± 0.1090.666 ± 0.0860.659 ± 0.067100.620 ± 0.1100.659 ± 0.1040.629 ± 0.1500.640 ± 0.1350.629 ± 0.1390.664 ± 0.0530.659 ± 0.035Sonar00.668 ± 0.0650.686 ± 0.1220.688 ± 0.1150.659 ± 0.0660.659 ± 0.0660.686 ± 0.0650.668 ± 0.08150.645 ± 0.0350.678 ± 0.0990.654 ± 0.1150.644 ± 0.0350.643 ± 0.0470.659 ± 0.0120.666 ± 0.109100.634 ± 0.1050.659 ± 0.1060.647 ± 0.1100.638 ± 0.0880.636 ± 0.0950.658 ± 0.0250.665 ± 0.053Average0.755 ± 0.0610.772 ± 0.0630.764 ± 0.0600.762 ± 0.0560.761 ± 0.0560.772 ± 0.0590.779 ± 0.067Table 4
Comparison of classification performance of different FRS models on LSVM in original and noisy data.
DatasetsNoise level (%)FRSβ-PFRSK-trimmed FRSK-means FRSK-median FRSSFRSRS-FRSGlass00.548 ± 0.1220.571 ± 0.1260.570 ± 0.1280.581 ± 0.1280.575 ± 0.1280.582 ± 0.1220.548 ± 0.12250.542 ± 0.0340.569 ± 0.0210.567 ± 0.0070.575 ± 0.0330.574 ± 0.0510.580 ± 0.0850.547 ± 0.090100.539 ± 0.0280.564 ± 0.0150.565 ± 0.0270.571 ± 0.0190.572 ± 0.0290.577 ± 0.0600.545 ± 0.110Wine00.938 ± 0.0550.950 ± 0.0720.950 ± 0.0720.938 ± 0.0550.938 ± 0.0550.948 ± 0.0550.949 ± 0.06850.892 ± 0.0620.945 ± 0.0550.936 ± 0.0790.937 ± 0.0390.935 ± 0.0500.946 ± 0.0340.947 ± 0.101100.890 ± 0.0640.940 ± 0.0270.935 ± 0.0700.933 ± 0.0290.935 ± 0.0560.941 ± 0.0820.942 ± 0.069Heart00.770 ± 0.0870.830 ± 0.0580.778 ± 0.0700.767 ± 0.0800.767 ± 0.0800.770 ± 0.0870.770 ± 0.08750.767 ± 0.0800.781 ± 0.0440.772 ± 0.0860.761 ± 0.0510.760 ± 0.0390.768 ± 0.0660.766 ± 0.043100.753 ± 0.0430.754 ± 0.0550.752 ± 0.0650.741 ± 0.0120.752 ± 0.0250.755 ± 0.0380.765 ± 0.071Segment00.903 ± 0.0550.904 ± 0.0540.904 ± 0.0530.905 ± 0.0520.904 ± 0.0530.905 ± 0.0520.905 ± 0.05250.894 ± 0.0570.896 ± 0.0180.901 ± 0.0870.895 ± 0.0100.895 ± 0.0710.899 ± 0.0250.903 ± 0.044100.874 ± 0.0200.876 ± 0.0420.875 ± 0.0590.877 ± 0.0760.878 ± 0.0930.894 ± 0.1010.897 ± 0.069Hepatitis00.815 ± 0.0690.815 ± 0.0800.815 ± 0.0800.815 ± 0.0800.815 ± 0.0800.845 ± 0.0800.845 ± 0.08050.805 ± 0.0570.811 ± 0.0360.809 ± 0.0610.814 ± 0.0270.813 ± 0.0540.832 ± 0.0990.834 ± 0.096100.793 ± 0.0910.801 ± 0.0610.815 ± 0.0450.812 ± 0.0520.811 ± 0.0590.833 ± 0.0670.833 ± 0.013ICU00.926 ± 0.0230.926 ± 0.0230.926 ± 0.0230.926 ± 0.0230.926 ± 0.0230.926 ± 0.0230.926 ± 0.02350.910 ± 0.0470.920 ± 0.0180.920 ± 0.0230.923 ± 0.0330.919 ± 0.0390.921 ± 0.0710.924 ± 0.104100.899 ± 0.0130.911 ± 0.0470.910 ± 0.0430.903 ± 0.0220.906 ± 0.0190.919 ± 0.0450.922 ± 0.055German00.735 ± 0.0540.739 ± 0.0570.739 ± 0.0570.742 ± 0.0550.742 ± 0.0550.732 ± 0.0540.735 ± 0.05450.720 ± 0.0550.722 ± 0.1040.725 ± 0.0460.725 ± 0.1150.729 ± 0.0480.727 ± 0.0150.731 ± 0.080100.701 ± 0.0470.714 ± 0.0540.711 ± 0.0480.702 ± 0.0440.708 ± 0.0410.719 ± 0.0390.725 ± 0.011Soy00.765 ± 0.0240.775 ± 0.0240.765 ± 0.0240.765 ± 0.0240.765 ± 0.0240.765 ± 0.0630.775 ± 0.02450.705 ± 0.0320.740 ± 0.0250.736 ± 0.0390.740 ± 0.0330.738 ± 0.0740.739 ± 0.0170.763 ± 0.117100.700 ± 0.0470.731 ± 0.0630.724 ± 0.0320.722 ± 0.0180.721 ± 0.0580.735 ± 0.0230.753 ± 0.027Horse00.832 ± 0.0520.829 ± 0.0520.842 ± 0.0570.842 ± 0.0570.834 ± 0.0530.839 ± 0.0520.832 ± 0.05250.826 ± 0.0590.828 ± 0.0690.834 ± 0.0530.835 ± 0.0440.831 ± 0.0340.834 ± 0.0730.831 ± 0.054100.807 ± 0.0440.824 ± 0.0680.832 ± 0.0570.823 ± 0.0500.821 ± 0.0590.829 ± 0.0260.830 ± 0.080WDBC00.935 ± 0.0350.943 ± 0.0200.935 ± 0.0460.933 ± 0.0470.939 ± 0.0470.935 ± 0.0350.935 ± 0.03550.930 ± 0.0420.942 ± 0.0250.933 ± 0.0320.931 ± 0.0350.933 ± 0.0390.934 ± 0.0590.934 ± 0.100100.923 ± 0.0400.933 ± 0.0210.932 ± 0.0440.930 ± 0.0450.926 ± 0.0470.931 ± 0.0610.933 ± 0.017WPBC00.763 ± 0.0300.763 ± 0.0300.763 ± 0.0300.763 ± 0.0300.763 ± 0.0300.763 ± 0.0300.763 ± 0.03050.711 ± 0.0270.716 ± 0.0150.721 ± 0.0520.734 ± 0.0240.733 ± 0.0680.762 ± 0.0800.734 ± 0.087100.705 ± 0.0820.714 ± 0.1090.720 ± 0.0760.724 ± 0.0640.730 ± 0.0990.759 ± 0.0550.730 ± 0.020Sonar00.712 ± 0.0720.725 ± 0.0710.716 ± 0.0890.721 ± 0.1100.721 ± 0.1100.722 ± 0.0720.733 ± 0.05550.711 ± 0.0690.719 ± 0.1010.714 ± 0.0760.717 ± 0.1190.716 ± 0.1190.715 ± 0.0680.728 ± 0.038100.673 ± 0.0260.716 ± 0.0120.712 ± 0.0340.711 ± 0.0970.713 ± 0.0250.713 ± 0.0240.718 ± 0.073Average0.786 ± 0.0510.801 ± 0.0490.799 ± 0.0550.798 ± 0.0510.798 ± 0.0570.805 ± 0.0570.804 ± 0.064In terms of the global perspective on the three classifiers, the average precision of the RS-FRS model is higher than that of the other FRS models upon the original data (0%) and the noise data (5% and 10%). Furthermore, in terms of the perspective of each dataset, each classifier, and each noise level, with KNN, RS-FRS model is optimal on 8 out of 12 datasets upon the original data (0%), and it is optimal on 15 out of 24 datasets upon the noise data (5% and 10%) in Table2. Similar results are shown in Tables 3 and 4. With CART, RS-FRS model is superior on 7 out of 12 datasets upon the original data (0%), and it also is superior on 16 out of 24 datasets upon the noise data (5% and 10%) in Table 3. With LSVM, RS-FRS model is optimal on 6 out of 12 datasets upon the original data (0%), and it is optimal on 16 out of 24 datasets upon the noise data (5% and 10%) in Table 4. It is worth noting that for wine, soy, hepatitis, ICU and WPBC, these 7 FRS models have the same or similar performance in 0% noise level. This is because there is no noise in the original data according to our obvious assumption; the 7 FRS models find the same sample as the nearest sample of the target sample to calculate the upper and lower approximations of the target sample.


#### 5.2.2. Robustness Analysis

In addition, after adding the noise gradually, the decreasing degree of the average precision of different models (Tables2–4) is used as the basis for comparing the robustness of these models. In Figures 1–3, FRS,β-PFRS, K-trimmed FRS, K-means FRS, K-median FRS, SFRS, and RS-FRS model are numbered from 1 to 7 in sequence.Figure 1
Comparison of robustness of different FRS models on KNN.Figure 2
Comparison of robustness of different FRS models on CART.Figure 3
Comparison of robustness of different FRS models on LSVM.As shown in Figures1–3, as the noise level increases gradually, the performance of every FRS model declines on three classifiers. This kind of phenomenon is consistent with the actual situation, because the noisy sample may be regarded as the nearest sample of the given target sample when these models calculate the lower approximation of the target sample. Therefore, the classification accuracy of these FRS models will reduce. However, among these models, the classical FRS model declines more sharply and changes more dramatically. This is because the classical FRS model is the most dependent on the nearest sample and it has the least robust performance. The proposed RS-FRS model presents the smallest decline and the least drastic change, indicating that our strategy that divides the sample set into fuzzy decision classes is in line with reality and has the best robust performance.


#### 5.2.3. Statistical Analysis

To further explore whether there are significant differences in the average classification performance of the seven FRS models, we adopted Friedman test [35] and Bonferroni-Dunn test [36]. The Friedman statistical coefficient is defined as(18)χF2=12Nkk+1∑i=1kRi2−kk+124,FF=N−1χF2Nk−1−χF2,where Ri is the average ranking of model i in all datasets and k is the total number of models and N is the number of datasets. FF obeys the Fisher distribution of k−1 and k−1N−1 degrees of freedom. In this paper, k=7 and N=36. At significance level α=0.1, F6,210=1.77. Table 5 shows the Friedman statistical coefficients and corresponding critical values under different classifiers. If FF is greater than the critical value, the null hypothesis is rejected, and we believe that there is a significant difference among the performance of all models. On the contrary, the null hypothesis cannot be rejected and it can be considered that there is no significant difference among the performances of all models.Table 5
Friedman statistical coefficientsFF and corresponding critical values.
ClassifierFFCritical value (α=0.1)KNN7.12681.77CART16.3475LSVM17.1893If Friedman’s null hypothesis is rejected, the Bonferroni-Dunn statistical verification method can further analyze the relative performance between each comparison model and RS-FRS model, and RS-FRS model is regarded as the control model. If there is a significant difference between the two models, the difference between the average ranking between them should be at least greater than the critical difference (CD):(19)CD=qαkk+16N,where k is the total number of models, N is the number of datasets, and qα is the critical value of Bonferroni-Dunn test at the corresponding significance level. At significance level α=0.1, qα=2.394. Therefore, CD=1.2190k=7,N=36. If the average ranking difference between the RS-FRS model and a comparison model is greater than the value of CD, we consider their performance to be significantly different.Figures4–6 show the CD diagram of the three classifiers, in which the average ranking of all algorithms is drawn in turn along the horizontal axis. In other words, scales from 1 to 7 on the horizontal axis are the average ranking of the seven algorithms over 12 datasets, with the higher the ranking, the better the algorithm, and the algorithm with the highest ranking (optimal) in the axis is located on the right side of the axis. If the average ranking between RS-FRS and a comparison algorithm is connected by a thick line (CD value), it means that RS-FRS has comparable performance with the comparison algorithm. If the average ranking difference between the RS-FRS model and a comparison model is greater than CD value, we consider their performance to be significantly different. As shown in the above figures, we can obtain the following results: (1) under the KNN, the RS-FRS is obviously better than the model of FRS, β-PFRS, K-trimmed FRS, K-means FRS, K-median FRS, and SFRS model Moreover, no other comparison model has the same performance as RS-FRS model in statistical verification. (2) Under the CART, the RS-FRS is obviously better than the model of FRS, β-PFRS, K-trimmed FRS, K-means FRS, K-median FRS, and SFRS model. No other comparison model has the same performance as RS-FRS model in statistical verification. (3) Under the LSVM, the RS-FRS and SFRS models are solid strong to the FRS, β-PFRS, K-trimmed FRS, K-means FRS, and K-median FRS model. Moreover, RS-FRS and SFRS models have considerable performance on statistics. Through the analysis of the above, the RS-FRS model performance is better than that of FRS, β-PFRS, K-trimmed FRS, K-means FRS, K-median FRS, and SFRS model.Figure 4
CD diagram of different FRS models on KNN based on Bonferroni-Dunn statistical method.Figure 5
CD diagram of different FRS models on CART based on Bonferroni-Dunn statistical method.Figure 6
CD diagram of different FRS models on LSVM based on Bonferroni-Dunn statistical method.


## 5.1. Datasets

In our experiments, 12 datasets from the open access UCI database [34] are utilized and described in Table 1. UCI dataset is a commonly used dataset of machine learning standard tests. For example, in dataset “wine”, these data are the results of a chemical analysis of wines grown in the same region in Italy but derived from three different cultivars. The analysis determined the quantities of 13 constituents found in each of the three types of wines. The attributes are alcohol, malic acid, ash, alkalinity of ash, magnesium, total phenols, flavanoids, nonflavanoid phenols, proanthocyanins, color intensity, hue, OD280/OD315 of diluted wines, and proline.Table 1
Descriptions of 12 datasets.
No.DatasetsSamplesAttributesClasses1Glass214962Wine1781333Heart2701324Segment23101875Hepatitis1551926ICU2002037German10002028Soy472149Horse36822210WDBC56930211WPBC19833212Sonar208602


## 5.2. Results

### 5.2.1. Comparison of Classification Accuracy

On three classifiers, we conduct comparison of classification accuracy with different FRS models in original data and noise data. The corresponding noise levels are 0%, 5%, and 10%, respectively. 0% noise level means the original data (the original data itself assumes no noise).5% (or 10%) noise level means there are 5% (or 10%) samples mislabeled randomly in the original data. In order to ensure the reliability of the experiments, we carry out the 10-fold cross-validation based on each independent noise random process. Tables2–4 show the comparison of classification performance of different FRS models on KNN, CART, and LSVM in original and noisy data. Bold values indicate the optimal performance among all FRS models, and italic values indicate the average classification accuracy of each model under all datasets.Table 2
Comparison of classification performance of different FRS models on KNN in original and noisy data.
DatasetsNoise level (%)FRSβ-PFRSK-trimmed FRSK-means FRSK-median FRSSFRSRS-FRSGlass00.678 ± 0.1340.650 ± 0.0680.631 ± 0.0610.650 ± 0.0780.655 ± 0.0990.678 ± 0.1340.668 ± 0.03150.629 ± 0.0610.649 ± 0.0270.630 ± 0.0370.647 ± 0.0910.648 ± 0.0430.655 ± 0.0960.656 ± 0.029100.628 ± 0.0980.647 ± 0.0660.626 ± 0.1100.645 ± 0.0190.644 ± 0.0680.648 ± 0.0890.649 ± 0.015Wine00.931 ± 0.0720.926 ± 0.0540.926 ± 0.0540.931 ± 0.0720.931 ± 0.0720.931 ± 0.0720.931 ± 0.07250.892 ± 0.0690.910 ± 0.0450.925 ± 0.0590.930 ± 0.0590.924 ± 0.0620.920 ± 0.0730.928 ± 0.086100.876 ± 0.0580.909 ± 0.0380.924 ± 0.0350.912 ± 0.0720.908 ± 0.0540.919 ± 0.0710.926 ± 0.044Heart00.744 ± 0.0690.759 ± 0.0820.741 ± 0.1050.730 ± 0.0780.730 ± 0.0780.744 ± 0.0690.744 ± 0.06950.735 ± 0.0710.739 ± 0.0590.740 ± 0.1100.728 ± 0.0800.727 ± 0.0130.741 ± 0.0670.742 ± 0.038100.719 ± 0.0820.733 ± 0.0860.734 ± 0.0440.726 ± 0.0190.727 ± 0.0780.736 ± 0.0540.736 ± 0.025Segment00.946 ± 0.1090.949 ± 0.1090.953 ± 0.1090.955 ± 0.1090.953 ± 0.1090.946 ± 0.0330.946 ± 0.10950.944 ± 0.0380.948 ± 0.0360.949 ± 0.0320.951 ± 0.0920.952 ± 0.0340.945 ± 0.0550.945 ± 0.010100.941 ± 0.0400.947 ± 0.0320.946 ± 0.0530.949 ± 0.0390.947 ± 0.0370.944 ± 0.0710.944 ± 0.104Hepatitis00.765 ± 0.1070.765 ± 0.1070.765 ± 0.1070.765 ± 0.1070.765 ± 0.1070.765 ± 0.1070.765 ± 0.10750.763 ± 0.0920.764 ± 0.1080.763 ± 0.0450.764 ± 0.0990.760 ± 0.1230.764 ± 0.0920.764 ± 0.062100.750 ± 0.1180.759 ± 0.0750.760 ± 0.0750.764 ± 0.0810.753 ± 0.0770.750 ± 0.0280.762 ± 0.104ICU00.868 ± 0.1620.879 ± 0.1830.879 ± 0.1830.868 ± 0.1290.868 ± 0.1290.868 ± 0.1620.880 ± 0.04250.856 ± 0.1760.859 ± 0.0870.860 ± 0.1580.857 ± 0.1620.855 ± 0.1460.861 ± 0.0540.863 ± 0.107100.850 ± 0.0230.855 ± 0.1260.853 ± 0.1010.856 ± 0.0230.852 ± 0.1760.856 ± 0.0100.862 ± 0.093German00.716 ± 0.0370.712 ± 0.0610.713 ± 0.0450.713 ± 0.0320.713 ± 0.0320.716 ± 0.0370.716 ± 0.03750.708 ± 0.0380.710 ± 0.0460.710 ± 0.0590.709 ± 0.0370.712 ± 0.0410.713 ± 0.0630.714 ± 0.031100.706 ± 0.0330.710 ± 0.0350.708 ± 0.0420.708 ± 0.0910.709 ± 0.0320.710 ± 0.0250.713 ± 0.088Soy00.850 ± 0.1410.825 ± 0.1340.850 ± 0.1410.850 ± 0.1410.850 ± 0.1410.850 ± 0.1410.850 ± 0.14150.790 ± 0.0620.819 ± 0.0770.818 ± 0.0250.818 ± 0.0180.816 ± 0.0500.820 ± 0.0410.820 ± 0.112100.760 ± 0.0460.818 ± 0.0820.810 ± 0.0820.806 ± 0.0870.809 ± 0.2460.816 ± 0.0920.818 ± 0.070Horse00.864 ± 0.0560.845 ± 0.0510.834 ± 0.0600.834 ± 0.0600.840 ± 0.0610.845 ± 0.0510.864 ± 0.05650.830 ± 0.0420.829 ± 0.0610.823 ± 0.0720.830 ± 0.0550.825 ± 0.0730.828 ± 0.0520.828 ± 0.081100.815 ± 0.0560.825 ± 0.0480.818 ± 0.0650.829 ± 0.0610.824 ± 0.0740.826 ± 0.0140.827 ± 0.099WDBC00.937 ± 0.0360.955 ± 0.0190.956 ± 0.0230.946 ± 0.0340.952 ± 0.0340.937 ± 0.0280.937 ± 0.03650.928 ± 0.0320.933 ± 0.0410.951 ± 0.0200.941 ± 0.0370.952 ± 0.0290.936 ± 0.0710.936 ± 0.111100.920 ± 0.0270.927 ± 0.0210.949 ± 0.0380.940 ± 0.0340.951 ± 0.0670.935 ± 0.0830.935 ± 0.027WPBC00.757 ± 0.0500.727 ± 0.0830.721 ± 0.1010.711 ± 0.0950.721 ± 0.0820.726 ± 0.1010.757 ± 0.05050.721 ± 0.0870.721 ± 0.0660.712 ± 0.1140.711 ± 0.0980.711 ± 0.0690.722 ± 0.0810.751 ± 0.114100.720 ± 0.0890.720 ± 0.0530.710 ± 0.0950.709 ± 0.1080.706 ± 0.1010.719 ± 0.0770.747 ± 0.093Sonar00.731 ± 0.0670.774 ± 0.0600.779 ± 0.0830.769 ± 0.0980.769 ± 0.0980.731 ± 0.0670.780 ± 0.09150.730 ± 0.0630.750 ± 0.0550.763 ± 0.0640.765 ± 0.1060.764 ± 0.0770.729 ± 0.0130.772 ± 0.123100.728 ± 0.0670.744 ± 0.0890.726 ± 0.0700.726 ± 0.0750.730 ± 0.0590.728 ± 0.0880.761 ± 0.105Average0.798 ± 0.0720.805 ± 0.0690.804 ± 0.0740.804 ± 0.0740.804 ± 0.0800.804 ± 0.0680.812 ± 0.073Table 3
Comparison of classification performance of different FRS models on CART in original and noisy data.
DatasetsNoise level (%)FRSβ-PFRSK-trimmed FRSK-means FRSK-median FRSSFRSRS-FRSGlass00.478 ± 0.0310.436 ± 0.0270.454 ± 0.0390.458 ± 0.0450.455 ± 0.0250.478 ± 0.0310.478 ± 0.03150.430 ± 0.0250.434 ± 0.0260.433 ± 0.0100.436 ± 0.0040.434 ± 0.0490.463 ± 0.0690.465 ± 0.060100.429 ± 0.0690.431 ± 0.0140.432 ± 0.0250.431 ± 0.0510.430 ± 0.0520.454 ± 0.0100.460 ± 0.025Wine00.909 ± 0.0680.909 ± 0.0680.908 ± 0.0810.908 ± 0.0810.908 ± 0.0810.908 ± 0.0810.909 ± 0.06850.859 ± 0.0590.903 ± 0.0920.901 ± 0.0260.860 ± 0.0750.899 ± 0.0640.902 ± 0.0720.903 ± 0.083100.819 ± 0.0760.857 ± 0.0370.897 ± 0.0870.859 ± 0.0410.893 ± 0.0560.896 ± 0.0540.902 ± 0.107Heart00.707 ± 0.0190.759 ± 0.0990.752 ± 0.0890.707 ± 0.0190.707 ± 0.0190.707 ± 0.1240.752 ± 0.05250.694 ± 0.0540.737 ± 0.0910.707 ± 0.0190.702 ± 0.0470.704 ± 0.0100.705 ± 0.0550.750 ± 0.049100.659 ± 0.0450.728 ± 0.0430.693 ± 0.0110.682 ± 0.0440.693 ± 0.0310.704 ± 0.0230.748 ± 0.076Segment00.947 ± 0.0290.946 ± 0.0390.945 ± 0.0350.945 ± 0.0350.945 ± 0.0350.947 ± 0.0290.947 ± 0.02950.941 ± 0.0310.953 ± 0.0270.942 ± 0.0410.943 ± 0.0340.942 ± 0.0270.945 ± 0.0830.946 ± 0.082100.938 ± 0.0360.938 ± 0.0300.940 ± 0.0320.941 ± 0.0350.941 ± 0.0380.942 ± 0.0640.944 ± 0.033Hepatitis00.800 ± 0.0630.800 ± 0.0630.800 ± 0.0630.800 ± 0.0630.800 ± 0.0630.800 ± 0.0630.800 ± 0.06350.762 ± 0.0930.761 ± 0.1030.770 ± 0.1300.753 ± 0.0810.755 ± 0.1010.796 ± 0.0390.798 ± 0.080100.723 ± 0.1260.752 ± 0.0790.732 ± 0.0210.751 ± 0.1040.728 ± 0.1200.781 ± 0.0410.790 ± 0.105ICU00.799 ± 0.0850.783 ± 0.0360.783 ± 0.0360.783 ± 0.0360.783 ± 0.0360.799 ± 0.0850.799 ± 0.08550.760 ± 0.0620.764 ± 0.0240.763 ± 0.0220.773 ± 0.0450.761 ± 0.0190.762 ± 0.0770.786 ± 0.091100.751 ± 0.0130.755 ± 0.0880.754 ± 0.0960.755 ± 0.0190.754 ± 0.0490.752 ± 0.0820.761 ± 0.114German00.716 ± 0.0370.704 ± 0.0350.713 ± 0.0450.713 ± 0.0320.713 ± 0.0320.712 ± 0.0460.716 ± 0.03750.716 ± 0.0530.700 ± 0.0310.711 ± 0.0390.712 ± 0.0340.710 ± 0.0380.710 ± 0.0710.713 ± 0.064100.707 ± 0.0330.700 ± 0.0610.709 ± 0.0310.708 ± 0.0420.709 ± 0.0320.710 ± 0.0790.709 ± 0.088Soy00.915 ± 0.1110.915 ± 0.1110.915 ± 0.1110.915 ± 0.1110.915 ± 0.1110.915 ± 0.1110.915 ± 0.11150.810 ± 0.0670.883 ± 0.0770.879 ± 0.0040.891 ± 0.0670.878 ± 0.0580.894 ± 0.0680.893 ± 0.108100.790 ± 0.0390.810 ± 0.0510.806 ± 0.0580.808 ± 0.0130.806 ± 0.0250.821 ± 0.0220.825 ± 0.024Horse00.855 ± 0.0460.862 ± 0.0490.857 ± 0.0500.854 ± 0.0470.857 ± 0.0500.856 ± 0.0520.855 ± 0.04650.846 ± 0.0670.860 ± 0.0530.854 ± 0.0600.853 ± 0.0500.852 ± 0.0470.855 ± 0.0130.855 ± 0.103100.841 ± 0.0470.848 ± 0.0550.848 ± 0.0380.851 ± 0.0330.853 ± 0.0400.853 ± 0.0510.854 ± 0.027WDBC00.907 ± 0.0370.931 ± 0.0460.907 ± 0.0420.926 ± 0.0440.916 ± 0.0380.907 ± 0.0370.928 ± 0.02950.903 ± 0.0840.924 ± 0.0310.903 ± 0.0410.909 ± 0.0330.914 ± 0.0340.905 ± 0.1020.927 ± 0.057100.895 ± 0.0580.923 ± 0.0220.905 ± 0.0450.907 ± 0.0320.896 ± 0.0360.904 ± 0.0460.924 ± 0.037WPBC00.660 ± 0.1150.703 ± 0.1280.645 ± 0.1090.660 ± 0.1340.634 ± 0.1590.666 ± 0.0900.660 ± 0.09250.630 ± 0.1180.702 ± 0.1120.640 ± 0.1050.656 ± 0.1530.631 ± 0.1090.666 ± 0.0860.659 ± 0.067100.620 ± 0.1100.659 ± 0.1040.629 ± 0.1500.640 ± 0.1350.629 ± 0.1390.664 ± 0.0530.659 ± 0.035Sonar00.668 ± 0.0650.686 ± 0.1220.688 ± 0.1150.659 ± 0.0660.659 ± 0.0660.686 ± 0.0650.668 ± 0.08150.645 ± 0.0350.678 ± 0.0990.654 ± 0.1150.644 ± 0.0350.643 ± 0.0470.659 ± 0.0120.666 ± 0.109100.634 ± 0.1050.659 ± 0.1060.647 ± 0.1100.638 ± 0.0880.636 ± 0.0950.658 ± 0.0250.665 ± 0.053Average0.755 ± 0.0610.772 ± 0.0630.764 ± 0.0600.762 ± 0.0560.761 ± 0.0560.772 ± 0.0590.779 ± 0.067Table 4
Comparison of classification performance of different FRS models on LSVM in original and noisy data.
DatasetsNoise level (%)FRSβ-PFRSK-trimmed FRSK-means FRSK-median FRSSFRSRS-FRSGlass00.548 ± 0.1220.571 ± 0.1260.570 ± 0.1280.581 ± 0.1280.575 ± 0.1280.582 ± 0.1220.548 ± 0.12250.542 ± 0.0340.569 ± 0.0210.567 ± 0.0070.575 ± 0.0330.574 ± 0.0510.580 ± 0.0850.547 ± 0.090100.539 ± 0.0280.564 ± 0.0150.565 ± 0.0270.571 ± 0.0190.572 ± 0.0290.577 ± 0.0600.545 ± 0.110Wine00.938 ± 0.0550.950 ± 0.0720.950 ± 0.0720.938 ± 0.0550.938 ± 0.0550.948 ± 0.0550.949 ± 0.06850.892 ± 0.0620.945 ± 0.0550.936 ± 0.0790.937 ± 0.0390.935 ± 0.0500.946 ± 0.0340.947 ± 0.101100.890 ± 0.0640.940 ± 0.0270.935 ± 0.0700.933 ± 0.0290.935 ± 0.0560.941 ± 0.0820.942 ± 0.069Heart00.770 ± 0.0870.830 ± 0.0580.778 ± 0.0700.767 ± 0.0800.767 ± 0.0800.770 ± 0.0870.770 ± 0.08750.767 ± 0.0800.781 ± 0.0440.772 ± 0.0860.761 ± 0.0510.760 ± 0.0390.768 ± 0.0660.766 ± 0.043100.753 ± 0.0430.754 ± 0.0550.752 ± 0.0650.741 ± 0.0120.752 ± 0.0250.755 ± 0.0380.765 ± 0.071Segment00.903 ± 0.0550.904 ± 0.0540.904 ± 0.0530.905 ± 0.0520.904 ± 0.0530.905 ± 0.0520.905 ± 0.05250.894 ± 0.0570.896 ± 0.0180.901 ± 0.0870.895 ± 0.0100.895 ± 0.0710.899 ± 0.0250.903 ± 0.044100.874 ± 0.0200.876 ± 0.0420.875 ± 0.0590.877 ± 0.0760.878 ± 0.0930.894 ± 0.1010.897 ± 0.069Hepatitis00.815 ± 0.0690.815 ± 0.0800.815 ± 0.0800.815 ± 0.0800.815 ± 0.0800.845 ± 0.0800.845 ± 0.08050.805 ± 0.0570.811 ± 0.0360.809 ± 0.0610.814 ± 0.0270.813 ± 0.0540.832 ± 0.0990.834 ± 0.096100.793 ± 0.0910.801 ± 0.0610.815 ± 0.0450.812 ± 0.0520.811 ± 0.0590.833 ± 0.0670.833 ± 0.013ICU00.926 ± 0.0230.926 ± 0.0230.926 ± 0.0230.926 ± 0.0230.926 ± 0.0230.926 ± 0.0230.926 ± 0.02350.910 ± 0.0470.920 ± 0.0180.920 ± 0.0230.923 ± 0.0330.919 ± 0.0390.921 ± 0.0710.924 ± 0.104100.899 ± 0.0130.911 ± 0.0470.910 ± 0.0430.903 ± 0.0220.906 ± 0.0190.919 ± 0.0450.922 ± 0.055German00.735 ± 0.0540.739 ± 0.0570.739 ± 0.0570.742 ± 0.0550.742 ± 0.0550.732 ± 0.0540.735 ± 0.05450.720 ± 0.0550.722 ± 0.1040.725 ± 0.0460.725 ± 0.1150.729 ± 0.0480.727 ± 0.0150.731 ± 0.080100.701 ± 0.0470.714 ± 0.0540.711 ± 0.0480.702 ± 0.0440.708 ± 0.0410.719 ± 0.0390.725 ± 0.011Soy00.765 ± 0.0240.775 ± 0.0240.765 ± 0.0240.765 ± 0.0240.765 ± 0.0240.765 ± 0.0630.775 ± 0.02450.705 ± 0.0320.740 ± 0.0250.736 ± 0.0390.740 ± 0.0330.738 ± 0.0740.739 ± 0.0170.763 ± 0.117100.700 ± 0.0470.731 ± 0.0630.724 ± 0.0320.722 ± 0.0180.721 ± 0.0580.735 ± 0.0230.753 ± 0.027Horse00.832 ± 0.0520.829 ± 0.0520.842 ± 0.0570.842 ± 0.0570.834 ± 0.0530.839 ± 0.0520.832 ± 0.05250.826 ± 0.0590.828 ± 0.0690.834 ± 0.0530.835 ± 0.0440.831 ± 0.0340.834 ± 0.0730.831 ± 0.054100.807 ± 0.0440.824 ± 0.0680.832 ± 0.0570.823 ± 0.0500.821 ± 0.0590.829 ± 0.0260.830 ± 0.080WDBC00.935 ± 0.0350.943 ± 0.0200.935 ± 0.0460.933 ± 0.0470.939 ± 0.0470.935 ± 0.0350.935 ± 0.03550.930 ± 0.0420.942 ± 0.0250.933 ± 0.0320.931 ± 0.0350.933 ± 0.0390.934 ± 0.0590.934 ± 0.100100.923 ± 0.0400.933 ± 0.0210.932 ± 0.0440.930 ± 0.0450.926 ± 0.0470.931 ± 0.0610.933 ± 0.017WPBC00.763 ± 0.0300.763 ± 0.0300.763 ± 0.0300.763 ± 0.0300.763 ± 0.0300.763 ± 0.0300.763 ± 0.03050.711 ± 0.0270.716 ± 0.0150.721 ± 0.0520.734 ± 0.0240.733 ± 0.0680.762 ± 0.0800.734 ± 0.087100.705 ± 0.0820.714 ± 0.1090.720 ± 0.0760.724 ± 0.0640.730 ± 0.0990.759 ± 0.0550.730 ± 0.020Sonar00.712 ± 0.0720.725 ± 0.0710.716 ± 0.0890.721 ± 0.1100.721 ± 0.1100.722 ± 0.0720.733 ± 0.05550.711 ± 0.0690.719 ± 0.1010.714 ± 0.0760.717 ± 0.1190.716 ± 0.1190.715 ± 0.0680.728 ± 0.038100.673 ± 0.0260.716 ± 0.0120.712 ± 0.0340.711 ± 0.0970.713 ± 0.0250.713 ± 0.0240.718 ± 0.073Average0.786 ± 0.0510.801 ± 0.0490.799 ± 0.0550.798 ± 0.0510.798 ± 0.0570.805 ± 0.0570.804 ± 0.064In terms of the global perspective on the three classifiers, the average precision of the RS-FRS model is higher than that of the other FRS models upon the original data (0%) and the noise data (5% and 10%). Furthermore, in terms of the perspective of each dataset, each classifier, and each noise level, with KNN, RS-FRS model is optimal on 8 out of 12 datasets upon the original data (0%), and it is optimal on 15 out of 24 datasets upon the noise data (5% and 10%) in Table2. Similar results are shown in Tables 3 and 4. With CART, RS-FRS model is superior on 7 out of 12 datasets upon the original data (0%), and it also is superior on 16 out of 24 datasets upon the noise data (5% and 10%) in Table 3. With LSVM, RS-FRS model is optimal on 6 out of 12 datasets upon the original data (0%), and it is optimal on 16 out of 24 datasets upon the noise data (5% and 10%) in Table 4. It is worth noting that for wine, soy, hepatitis, ICU and WPBC, these 7 FRS models have the same or similar performance in 0% noise level. This is because there is no noise in the original data according to our obvious assumption; the 7 FRS models find the same sample as the nearest sample of the target sample to calculate the upper and lower approximations of the target sample.


### 5.2.2. Robustness Analysis

In addition, after adding the noise gradually, the decreasing degree of the average precision of different models (Tables2–4) is used as the basis for comparing the robustness of these models. In Figures 1–3, FRS,β-PFRS, K-trimmed FRS, K-means FRS, K-median FRS, SFRS, and RS-FRS model are numbered from 1 to 7 in sequence.Figure 1
Comparison of robustness of different FRS models on KNN.Figure 2
Comparison of robustness of different FRS models on CART.Figure 3
Comparison of robustness of different FRS models on LSVM.As shown in Figures1–3, as the noise level increases gradually, the performance of every FRS model declines on three classifiers. This kind of phenomenon is consistent with the actual situation, because the noisy sample may be regarded as the nearest sample of the given target sample when these models calculate the lower approximation of the target sample. Therefore, the classification accuracy of these FRS models will reduce. However, among these models, the classical FRS model declines more sharply and changes more dramatically. This is because the classical FRS model is the most dependent on the nearest sample and it has the least robust performance. The proposed RS-FRS model presents the smallest decline and the least drastic change, indicating that our strategy that divides the sample set into fuzzy decision classes is in line with reality and has the best robust performance.


### 5.2.3. Statistical Analysis

To further explore whether there are significant differences in the average classification performance of the seven FRS models, we adopted Friedman test [35] and Bonferroni-Dunn test [36]. The Friedman statistical coefficient is defined as(18)χF2=12Nkk+1∑i=1kRi2−kk+124,FF=N−1χF2Nk−1−χF2,where Ri is the average ranking of model i in all datasets and k is the total number of models and N is the number of datasets. FF obeys the Fisher distribution of k−1 and k−1N−1 degrees of freedom. In this paper, k=7 and N=36. At significance level α=0.1, F6,210=1.77. Table 5 shows the Friedman statistical coefficients and corresponding critical values under different classifiers. If FF is greater than the critical value, the null hypothesis is rejected, and we believe that there is a significant difference among the performance of all models. On the contrary, the null hypothesis cannot be rejected and it can be considered that there is no significant difference among the performances of all models.Table 5
Friedman statistical coefficientsFF and corresponding critical values.
ClassifierFFCritical value (α=0.1)KNN7.12681.77CART16.3475LSVM17.1893If Friedman’s null hypothesis is rejected, the Bonferroni-Dunn statistical verification method can further analyze the relative performance between each comparison model and RS-FRS model, and RS-FRS model is regarded as the control model. If there is a significant difference between the two models, the difference between the average ranking between them should be at least greater than the critical difference (CD):(19)CD=qαkk+16N,where k is the total number of models, N is the number of datasets, and qα is the critical value of Bonferroni-Dunn test at the corresponding significance level. At significance level α=0.1, qα=2.394. Therefore, CD=1.2190k=7,N=36. If the average ranking difference between the RS-FRS model and a comparison model is greater than the value of CD, we consider their performance to be significantly different.Figures4–6 show the CD diagram of the three classifiers, in which the average ranking of all algorithms is drawn in turn along the horizontal axis. In other words, scales from 1 to 7 on the horizontal axis are the average ranking of the seven algorithms over 12 datasets, with the higher the ranking, the better the algorithm, and the algorithm with the highest ranking (optimal) in the axis is located on the right side of the axis. If the average ranking between RS-FRS and a comparison algorithm is connected by a thick line (CD value), it means that RS-FRS has comparable performance with the comparison algorithm. If the average ranking difference between the RS-FRS model and a comparison model is greater than CD value, we consider their performance to be significantly different. As shown in the above figures, we can obtain the following results: (1) under the KNN, the RS-FRS is obviously better than the model of FRS, β-PFRS, K-trimmed FRS, K-means FRS, K-median FRS, and SFRS model Moreover, no other comparison model has the same performance as RS-FRS model in statistical verification. (2) Under the CART, the RS-FRS is obviously better than the model of FRS, β-PFRS, K-trimmed FRS, K-means FRS, K-median FRS, and SFRS model. No other comparison model has the same performance as RS-FRS model in statistical verification. (3) Under the LSVM, the RS-FRS and SFRS models are solid strong to the FRS, β-PFRS, K-trimmed FRS, K-means FRS, and K-median FRS model. Moreover, RS-FRS and SFRS models have considerable performance on statistics. Through the analysis of the above, the RS-FRS model performance is better than that of FRS, β-PFRS, K-trimmed FRS, K-means FRS, K-median FRS, and SFRS model.Figure 4
CD diagram of different FRS models on KNN based on Bonferroni-Dunn statistical method.Figure 5
CD diagram of different FRS models on CART based on Bonferroni-Dunn statistical method.Figure 6
CD diagram of different FRS models on LSVM based on Bonferroni-Dunn statistical method.


## 5.2.1. Comparison of Classification Accuracy

On three classifiers, we conduct comparison of classification accuracy with different FRS models in original data and noise data. The corresponding noise levels are 0%, 5%, and 10%, respectively. 0% noise level means the original data (the original data itself assumes no noise).5% (or 10%) noise level means there are 5% (or 10%) samples mislabeled randomly in the original data. In order to ensure the reliability of the experiments, we carry out the 10-fold cross-validation based on each independent noise random process. Tables2–4 show the comparison of classification performance of different FRS models on KNN, CART, and LSVM in original and noisy data. Bold values indicate the optimal performance among all FRS models, and italic values indicate the average classification accuracy of each model under all datasets.Table 2
Comparison of classification performance of different FRS models on KNN in original and noisy data.
DatasetsNoise level (%)FRSβ-PFRSK-trimmed FRSK-means FRSK-median FRSSFRSRS-FRSGlass00.678 ± 0.1340.650 ± 0.0680.631 ± 0.0610.650 ± 0.0780.655 ± 0.0990.678 ± 0.1340.668 ± 0.03150.629 ± 0.0610.649 ± 0.0270.630 ± 0.0370.647 ± 0.0910.648 ± 0.0430.655 ± 0.0960.656 ± 0.029100.628 ± 0.0980.647 ± 0.0660.626 ± 0.1100.645 ± 0.0190.644 ± 0.0680.648 ± 0.0890.649 ± 0.015Wine00.931 ± 0.0720.926 ± 0.0540.926 ± 0.0540.931 ± 0.0720.931 ± 0.0720.931 ± 0.0720.931 ± 0.07250.892 ± 0.0690.910 ± 0.0450.925 ± 0.0590.930 ± 0.0590.924 ± 0.0620.920 ± 0.0730.928 ± 0.086100.876 ± 0.0580.909 ± 0.0380.924 ± 0.0350.912 ± 0.0720.908 ± 0.0540.919 ± 0.0710.926 ± 0.044Heart00.744 ± 0.0690.759 ± 0.0820.741 ± 0.1050.730 ± 0.0780.730 ± 0.0780.744 ± 0.0690.744 ± 0.06950.735 ± 0.0710.739 ± 0.0590.740 ± 0.1100.728 ± 0.0800.727 ± 0.0130.741 ± 0.0670.742 ± 0.038100.719 ± 0.0820.733 ± 0.0860.734 ± 0.0440.726 ± 0.0190.727 ± 0.0780.736 ± 0.0540.736 ± 0.025Segment00.946 ± 0.1090.949 ± 0.1090.953 ± 0.1090.955 ± 0.1090.953 ± 0.1090.946 ± 0.0330.946 ± 0.10950.944 ± 0.0380.948 ± 0.0360.949 ± 0.0320.951 ± 0.0920.952 ± 0.0340.945 ± 0.0550.945 ± 0.010100.941 ± 0.0400.947 ± 0.0320.946 ± 0.0530.949 ± 0.0390.947 ± 0.0370.944 ± 0.0710.944 ± 0.104Hepatitis00.765 ± 0.1070.765 ± 0.1070.765 ± 0.1070.765 ± 0.1070.765 ± 0.1070.765 ± 0.1070.765 ± 0.10750.763 ± 0.0920.764 ± 0.1080.763 ± 0.0450.764 ± 0.0990.760 ± 0.1230.764 ± 0.0920.764 ± 0.062100.750 ± 0.1180.759 ± 0.0750.760 ± 0.0750.764 ± 0.0810.753 ± 0.0770.750 ± 0.0280.762 ± 0.104ICU00.868 ± 0.1620.879 ± 0.1830.879 ± 0.1830.868 ± 0.1290.868 ± 0.1290.868 ± 0.1620.880 ± 0.04250.856 ± 0.1760.859 ± 0.0870.860 ± 0.1580.857 ± 0.1620.855 ± 0.1460.861 ± 0.0540.863 ± 0.107100.850 ± 0.0230.855 ± 0.1260.853 ± 0.1010.856 ± 0.0230.852 ± 0.1760.856 ± 0.0100.862 ± 0.093German00.716 ± 0.0370.712 ± 0.0610.713 ± 0.0450.713 ± 0.0320.713 ± 0.0320.716 ± 0.0370.716 ± 0.03750.708 ± 0.0380.710 ± 0.0460.710 ± 0.0590.709 ± 0.0370.712 ± 0.0410.713 ± 0.0630.714 ± 0.031100.706 ± 0.0330.710 ± 0.0350.708 ± 0.0420.708 ± 0.0910.709 ± 0.0320.710 ± 0.0250.713 ± 0.088Soy00.850 ± 0.1410.825 ± 0.1340.850 ± 0.1410.850 ± 0.1410.850 ± 0.1410.850 ± 0.1410.850 ± 0.14150.790 ± 0.0620.819 ± 0.0770.818 ± 0.0250.818 ± 0.0180.816 ± 0.0500.820 ± 0.0410.820 ± 0.112100.760 ± 0.0460.818 ± 0.0820.810 ± 0.0820.806 ± 0.0870.809 ± 0.2460.816 ± 0.0920.818 ± 0.070Horse00.864 ± 0.0560.845 ± 0.0510.834 ± 0.0600.834 ± 0.0600.840 ± 0.0610.845 ± 0.0510.864 ± 0.05650.830 ± 0.0420.829 ± 0.0610.823 ± 0.0720.830 ± 0.0550.825 ± 0.0730.828 ± 0.0520.828 ± 0.081100.815 ± 0.0560.825 ± 0.0480.818 ± 0.0650.829 ± 0.0610.824 ± 0.0740.826 ± 0.0140.827 ± 0.099WDBC00.937 ± 0.0360.955 ± 0.0190.956 ± 0.0230.946 ± 0.0340.952 ± 0.0340.937 ± 0.0280.937 ± 0.03650.928 ± 0.0320.933 ± 0.0410.951 ± 0.0200.941 ± 0.0370.952 ± 0.0290.936 ± 0.0710.936 ± 0.111100.920 ± 0.0270.927 ± 0.0210.949 ± 0.0380.940 ± 0.0340.951 ± 0.0670.935 ± 0.0830.935 ± 0.027WPBC00.757 ± 0.0500.727 ± 0.0830.721 ± 0.1010.711 ± 0.0950.721 ± 0.0820.726 ± 0.1010.757 ± 0.05050.721 ± 0.0870.721 ± 0.0660.712 ± 0.1140.711 ± 0.0980.711 ± 0.0690.722 ± 0.0810.751 ± 0.114100.720 ± 0.0890.720 ± 0.0530.710 ± 0.0950.709 ± 0.1080.706 ± 0.1010.719 ± 0.0770.747 ± 0.093Sonar00.731 ± 0.0670.774 ± 0.0600.779 ± 0.0830.769 ± 0.0980.769 ± 0.0980.731 ± 0.0670.780 ± 0.09150.730 ± 0.0630.750 ± 0.0550.763 ± 0.0640.765 ± 0.1060.764 ± 0.0770.729 ± 0.0130.772 ± 0.123100.728 ± 0.0670.744 ± 0.0890.726 ± 0.0700.726 ± 0.0750.730 ± 0.0590.728 ± 0.0880.761 ± 0.105Average0.798 ± 0.0720.805 ± 0.0690.804 ± 0.0740.804 ± 0.0740.804 ± 0.0800.804 ± 0.0680.812 ± 0.073Table 3
Comparison of classification performance of different FRS models on CART in original and noisy data.
DatasetsNoise level (%)FRSβ-PFRSK-trimmed FRSK-means FRSK-median FRSSFRSRS-FRSGlass00.478 ± 0.0310.436 ± 0.0270.454 ± 0.0390.458 ± 0.0450.455 ± 0.0250.478 ± 0.0310.478 ± 0.03150.430 ± 0.0250.434 ± 0.0260.433 ± 0.0100.436 ± 0.0040.434 ± 0.0490.463 ± 0.0690.465 ± 0.060100.429 ± 0.0690.431 ± 0.0140.432 ± 0.0250.431 ± 0.0510.430 ± 0.0520.454 ± 0.0100.460 ± 0.025Wine00.909 ± 0.0680.909 ± 0.0680.908 ± 0.0810.908 ± 0.0810.908 ± 0.0810.908 ± 0.0810.909 ± 0.06850.859 ± 0.0590.903 ± 0.0920.901 ± 0.0260.860 ± 0.0750.899 ± 0.0640.902 ± 0.0720.903 ± 0.083100.819 ± 0.0760.857 ± 0.0370.897 ± 0.0870.859 ± 0.0410.893 ± 0.0560.896 ± 0.0540.902 ± 0.107Heart00.707 ± 0.0190.759 ± 0.0990.752 ± 0.0890.707 ± 0.0190.707 ± 0.0190.707 ± 0.1240.752 ± 0.05250.694 ± 0.0540.737 ± 0.0910.707 ± 0.0190.702 ± 0.0470.704 ± 0.0100.705 ± 0.0550.750 ± 0.049100.659 ± 0.0450.728 ± 0.0430.693 ± 0.0110.682 ± 0.0440.693 ± 0.0310.704 ± 0.0230.748 ± 0.076Segment00.947 ± 0.0290.946 ± 0.0390.945 ± 0.0350.945 ± 0.0350.945 ± 0.0350.947 ± 0.0290.947 ± 0.02950.941 ± 0.0310.953 ± 0.0270.942 ± 0.0410.943 ± 0.0340.942 ± 0.0270.945 ± 0.0830.946 ± 0.082100.938 ± 0.0360.938 ± 0.0300.940 ± 0.0320.941 ± 0.0350.941 ± 0.0380.942 ± 0.0640.944 ± 0.033Hepatitis00.800 ± 0.0630.800 ± 0.0630.800 ± 0.0630.800 ± 0.0630.800 ± 0.0630.800 ± 0.0630.800 ± 0.06350.762 ± 0.0930.761 ± 0.1030.770 ± 0.1300.753 ± 0.0810.755 ± 0.1010.796 ± 0.0390.798 ± 0.080100.723 ± 0.1260.752 ± 0.0790.732 ± 0.0210.751 ± 0.1040.728 ± 0.1200.781 ± 0.0410.790 ± 0.105ICU00.799 ± 0.0850.783 ± 0.0360.783 ± 0.0360.783 ± 0.0360.783 ± 0.0360.799 ± 0.0850.799 ± 0.08550.760 ± 0.0620.764 ± 0.0240.763 ± 0.0220.773 ± 0.0450.761 ± 0.0190.762 ± 0.0770.786 ± 0.091100.751 ± 0.0130.755 ± 0.0880.754 ± 0.0960.755 ± 0.0190.754 ± 0.0490.752 ± 0.0820.761 ± 0.114German00.716 ± 0.0370.704 ± 0.0350.713 ± 0.0450.713 ± 0.0320.713 ± 0.0320.712 ± 0.0460.716 ± 0.03750.716 ± 0.0530.700 ± 0.0310.711 ± 0.0390.712 ± 0.0340.710 ± 0.0380.710 ± 0.0710.713 ± 0.064100.707 ± 0.0330.700 ± 0.0610.709 ± 0.0310.708 ± 0.0420.709 ± 0.0320.710 ± 0.0790.709 ± 0.088Soy00.915 ± 0.1110.915 ± 0.1110.915 ± 0.1110.915 ± 0.1110.915 ± 0.1110.915 ± 0.1110.915 ± 0.11150.810 ± 0.0670.883 ± 0.0770.879 ± 0.0040.891 ± 0.0670.878 ± 0.0580.894 ± 0.0680.893 ± 0.108100.790 ± 0.0390.810 ± 0.0510.806 ± 0.0580.808 ± 0.0130.806 ± 0.0250.821 ± 0.0220.825 ± 0.024Horse00.855 ± 0.0460.862 ± 0.0490.857 ± 0.0500.854 ± 0.0470.857 ± 0.0500.856 ± 0.0520.855 ± 0.04650.846 ± 0.0670.860 ± 0.0530.854 ± 0.0600.853 ± 0.0500.852 ± 0.0470.855 ± 0.0130.855 ± 0.103100.841 ± 0.0470.848 ± 0.0550.848 ± 0.0380.851 ± 0.0330.853 ± 0.0400.853 ± 0.0510.854 ± 0.027WDBC00.907 ± 0.0370.931 ± 0.0460.907 ± 0.0420.926 ± 0.0440.916 ± 0.0380.907 ± 0.0370.928 ± 0.02950.903 ± 0.0840.924 ± 0.0310.903 ± 0.0410.909 ± 0.0330.914 ± 0.0340.905 ± 0.1020.927 ± 0.057100.895 ± 0.0580.923 ± 0.0220.905 ± 0.0450.907 ± 0.0320.896 ± 0.0360.904 ± 0.0460.924 ± 0.037WPBC00.660 ± 0.1150.703 ± 0.1280.645 ± 0.1090.660 ± 0.1340.634 ± 0.1590.666 ± 0.0900.660 ± 0.09250.630 ± 0.1180.702 ± 0.1120.640 ± 0.1050.656 ± 0.1530.631 ± 0.1090.666 ± 0.0860.659 ± 0.067100.620 ± 0.1100.659 ± 0.1040.629 ± 0.1500.640 ± 0.1350.629 ± 0.1390.664 ± 0.0530.659 ± 0.035Sonar00.668 ± 0.0650.686 ± 0.1220.688 ± 0.1150.659 ± 0.0660.659 ± 0.0660.686 ± 0.0650.668 ± 0.08150.645 ± 0.0350.678 ± 0.0990.654 ± 0.1150.644 ± 0.0350.643 ± 0.0470.659 ± 0.0120.666 ± 0.109100.634 ± 0.1050.659 ± 0.1060.647 ± 0.1100.638 ± 0.0880.636 ± 0.0950.658 ± 0.0250.665 ± 0.053Average0.755 ± 0.0610.772 ± 0.0630.764 ± 0.0600.762 ± 0.0560.761 ± 0.0560.772 ± 0.0590.779 ± 0.067Table 4
Comparison of classification performance of different FRS models on LSVM in original and noisy data.
DatasetsNoise level (%)FRSβ-PFRSK-trimmed FRSK-means FRSK-median FRSSFRSRS-FRSGlass00.548 ± 0.1220.571 ± 0.1260.570 ± 0.1280.581 ± 0.1280.575 ± 0.1280.582 ± 0.1220.548 ± 0.12250.542 ± 0.0340.569 ± 0.0210.567 ± 0.0070.575 ± 0.0330.574 ± 0.0510.580 ± 0.0850.547 ± 0.090100.539 ± 0.0280.564 ± 0.0150.565 ± 0.0270.571 ± 0.0190.572 ± 0.0290.577 ± 0.0600.545 ± 0.110Wine00.938 ± 0.0550.950 ± 0.0720.950 ± 0.0720.938 ± 0.0550.938 ± 0.0550.948 ± 0.0550.949 ± 0.06850.892 ± 0.0620.945 ± 0.0550.936 ± 0.0790.937 ± 0.0390.935 ± 0.0500.946 ± 0.0340.947 ± 0.101100.890 ± 0.0640.940 ± 0.0270.935 ± 0.0700.933 ± 0.0290.935 ± 0.0560.941 ± 0.0820.942 ± 0.069Heart00.770 ± 0.0870.830 ± 0.0580.778 ± 0.0700.767 ± 0.0800.767 ± 0.0800.770 ± 0.0870.770 ± 0.08750.767 ± 0.0800.781 ± 0.0440.772 ± 0.0860.761 ± 0.0510.760 ± 0.0390.768 ± 0.0660.766 ± 0.043100.753 ± 0.0430.754 ± 0.0550.752 ± 0.0650.741 ± 0.0120.752 ± 0.0250.755 ± 0.0380.765 ± 0.071Segment00.903 ± 0.0550.904 ± 0.0540.904 ± 0.0530.905 ± 0.0520.904 ± 0.0530.905 ± 0.0520.905 ± 0.05250.894 ± 0.0570.896 ± 0.0180.901 ± 0.0870.895 ± 0.0100.895 ± 0.0710.899 ± 0.0250.903 ± 0.044100.874 ± 0.0200.876 ± 0.0420.875 ± 0.0590.877 ± 0.0760.878 ± 0.0930.894 ± 0.1010.897 ± 0.069Hepatitis00.815 ± 0.0690.815 ± 0.0800.815 ± 0.0800.815 ± 0.0800.815 ± 0.0800.845 ± 0.0800.845 ± 0.08050.805 ± 0.0570.811 ± 0.0360.809 ± 0.0610.814 ± 0.0270.813 ± 0.0540.832 ± 0.0990.834 ± 0.096100.793 ± 0.0910.801 ± 0.0610.815 ± 0.0450.812 ± 0.0520.811 ± 0.0590.833 ± 0.0670.833 ± 0.013ICU00.926 ± 0.0230.926 ± 0.0230.926 ± 0.0230.926 ± 0.0230.926 ± 0.0230.926 ± 0.0230.926 ± 0.02350.910 ± 0.0470.920 ± 0.0180.920 ± 0.0230.923 ± 0.0330.919 ± 0.0390.921 ± 0.0710.924 ± 0.104100.899 ± 0.0130.911 ± 0.0470.910 ± 0.0430.903 ± 0.0220.906 ± 0.0190.919 ± 0.0450.922 ± 0.055German00.735 ± 0.0540.739 ± 0.0570.739 ± 0.0570.742 ± 0.0550.742 ± 0.0550.732 ± 0.0540.735 ± 0.05450.720 ± 0.0550.722 ± 0.1040.725 ± 0.0460.725 ± 0.1150.729 ± 0.0480.727 ± 0.0150.731 ± 0.080100.701 ± 0.0470.714 ± 0.0540.711 ± 0.0480.702 ± 0.0440.708 ± 0.0410.719 ± 0.0390.725 ± 0.011Soy00.765 ± 0.0240.775 ± 0.0240.765 ± 0.0240.765 ± 0.0240.765 ± 0.0240.765 ± 0.0630.775 ± 0.02450.705 ± 0.0320.740 ± 0.0250.736 ± 0.0390.740 ± 0.0330.738 ± 0.0740.739 ± 0.0170.763 ± 0.117100.700 ± 0.0470.731 ± 0.0630.724 ± 0.0320.722 ± 0.0180.721 ± 0.0580.735 ± 0.0230.753 ± 0.027Horse00.832 ± 0.0520.829 ± 0.0520.842 ± 0.0570.842 ± 0.0570.834 ± 0.0530.839 ± 0.0520.832 ± 0.05250.826 ± 0.0590.828 ± 0.0690.834 ± 0.0530.835 ± 0.0440.831 ± 0.0340.834 ± 0.0730.831 ± 0.054100.807 ± 0.0440.824 ± 0.0680.832 ± 0.0570.823 ± 0.0500.821 ± 0.0590.829 ± 0.0260.830 ± 0.080WDBC00.935 ± 0.0350.943 ± 0.0200.935 ± 0.0460.933 ± 0.0470.939 ± 0.0470.935 ± 0.0350.935 ± 0.03550.930 ± 0.0420.942 ± 0.0250.933 ± 0.0320.931 ± 0.0350.933 ± 0.0390.934 ± 0.0590.934 ± 0.100100.923 ± 0.0400.933 ± 0.0210.932 ± 0.0440.930 ± 0.0450.926 ± 0.0470.931 ± 0.0610.933 ± 0.017WPBC00.763 ± 0.0300.763 ± 0.0300.763 ± 0.0300.763 ± 0.0300.763 ± 0.0300.763 ± 0.0300.763 ± 0.03050.711 ± 0.0270.716 ± 0.0150.721 ± 0.0520.734 ± 0.0240.733 ± 0.0680.762 ± 0.0800.734 ± 0.087100.705 ± 0.0820.714 ± 0.1090.720 ± 0.0760.724 ± 0.0640.730 ± 0.0990.759 ± 0.0550.730 ± 0.020Sonar00.712 ± 0.0720.725 ± 0.0710.716 ± 0.0890.721 ± 0.1100.721 ± 0.1100.722 ± 0.0720.733 ± 0.05550.711 ± 0.0690.719 ± 0.1010.714 ± 0.0760.717 ± 0.1190.716 ± 0.1190.715 ± 0.0680.728 ± 0.038100.673 ± 0.0260.716 ± 0.0120.712 ± 0.0340.711 ± 0.0970.713 ± 0.0250.713 ± 0.0240.718 ± 0.073Average0.786 ± 0.0510.801 ± 0.0490.799 ± 0.0550.798 ± 0.0510.798 ± 0.0570.805 ± 0.0570.804 ± 0.064In terms of the global perspective on the three classifiers, the average precision of the RS-FRS model is higher than that of the other FRS models upon the original data (0%) and the noise data (5% and 10%). Furthermore, in terms of the perspective of each dataset, each classifier, and each noise level, with KNN, RS-FRS model is optimal on 8 out of 12 datasets upon the original data (0%), and it is optimal on 15 out of 24 datasets upon the noise data (5% and 10%) in Table2. Similar results are shown in Tables 3 and 4. With CART, RS-FRS model is superior on 7 out of 12 datasets upon the original data (0%), and it also is superior on 16 out of 24 datasets upon the noise data (5% and 10%) in Table 3. With LSVM, RS-FRS model is optimal on 6 out of 12 datasets upon the original data (0%), and it is optimal on 16 out of 24 datasets upon the noise data (5% and 10%) in Table 4. It is worth noting that for wine, soy, hepatitis, ICU and WPBC, these 7 FRS models have the same or similar performance in 0% noise level. This is because there is no noise in the original data according to our obvious assumption; the 7 FRS models find the same sample as the nearest sample of the target sample to calculate the upper and lower approximations of the target sample.


## 5.2.2. Robustness Analysis

In addition, after adding the noise gradually, the decreasing degree of the average precision of different models (Tables2–4) is used as the basis for comparing the robustness of these models. In Figures 1–3, FRS,β-PFRS, K-trimmed FRS, K-means FRS, K-median FRS, SFRS, and RS-FRS model are numbered from 1 to 7 in sequence.Figure 1
Comparison of robustness of different FRS models on KNN.Figure 2
Comparison of robustness of different FRS models on CART.Figure 3
Comparison of robustness of different FRS models on LSVM.As shown in Figures1–3, as the noise level increases gradually, the performance of every FRS model declines on three classifiers. This kind of phenomenon is consistent with the actual situation, because the noisy sample may be regarded as the nearest sample of the given target sample when these models calculate the lower approximation of the target sample. Therefore, the classification accuracy of these FRS models will reduce. However, among these models, the classical FRS model declines more sharply and changes more dramatically. This is because the classical FRS model is the most dependent on the nearest sample and it has the least robust performance. The proposed RS-FRS model presents the smallest decline and the least drastic change, indicating that our strategy that divides the sample set into fuzzy decision classes is in line with reality and has the best robust performance.


## 5.2.3. Statistical Analysis

To further explore whether there are significant differences in the average classification performance of the seven FRS models, we adopted Friedman test [35] and Bonferroni-Dunn test [36]. The Friedman statistical coefficient is defined as(18)χF2=12Nkk+1∑i=1kRi2−kk+124,FF=N−1χF2Nk−1−χF2,where Ri is the average ranking of model i in all datasets and k is the total number of models and N is the number of datasets. FF obeys the Fisher distribution of k−1 and k−1N−1 degrees of freedom. In this paper, k=7 and N=36. At significance level α=0.1, F6,210=1.77. Table 5 shows the Friedman statistical coefficients and corresponding critical values under different classifiers. If FF is greater than the critical value, the null hypothesis is rejected, and we believe that there is a significant difference among the performance of all models. On the contrary, the null hypothesis cannot be rejected and it can be considered that there is no significant difference among the performances of all models.Table 5
Friedman statistical coefficientsFF and corresponding critical values.
ClassifierFFCritical value (α=0.1)KNN7.12681.77CART16.3475LSVM17.1893If Friedman’s null hypothesis is rejected, the Bonferroni-Dunn statistical verification method can further analyze the relative performance between each comparison model and RS-FRS model, and RS-FRS model is regarded as the control model. If there is a significant difference between the two models, the difference between the average ranking between them should be at least greater than the critical difference (CD):(19)CD=qαkk+16N,where k is the total number of models, N is the number of datasets, and qα is the critical value of Bonferroni-Dunn test at the corresponding significance level. At significance level α=0.1, qα=2.394. Therefore, CD=1.2190k=7,N=36. If the average ranking difference between the RS-FRS model and a comparison model is greater than the value of CD, we consider their performance to be significantly different.Figures4–6 show the CD diagram of the three classifiers, in which the average ranking of all algorithms is drawn in turn along the horizontal axis. In other words, scales from 1 to 7 on the horizontal axis are the average ranking of the seven algorithms over 12 datasets, with the higher the ranking, the better the algorithm, and the algorithm with the highest ranking (optimal) in the axis is located on the right side of the axis. If the average ranking between RS-FRS and a comparison algorithm is connected by a thick line (CD value), it means that RS-FRS has comparable performance with the comparison algorithm. If the average ranking difference between the RS-FRS model and a comparison model is greater than CD value, we consider their performance to be significantly different. As shown in the above figures, we can obtain the following results: (1) under the KNN, the RS-FRS is obviously better than the model of FRS, β-PFRS, K-trimmed FRS, K-means FRS, K-median FRS, and SFRS model Moreover, no other comparison model has the same performance as RS-FRS model in statistical verification. (2) Under the CART, the RS-FRS is obviously better than the model of FRS, β-PFRS, K-trimmed FRS, K-means FRS, K-median FRS, and SFRS model. No other comparison model has the same performance as RS-FRS model in statistical verification. (3) Under the LSVM, the RS-FRS and SFRS models are solid strong to the FRS, β-PFRS, K-trimmed FRS, K-means FRS, and K-median FRS model. Moreover, RS-FRS and SFRS models have considerable performance on statistics. Through the analysis of the above, the RS-FRS model performance is better than that of FRS, β-PFRS, K-trimmed FRS, K-means FRS, K-median FRS, and SFRS model.Figure 4
CD diagram of different FRS models on KNN based on Bonferroni-Dunn statistical method.Figure 5
CD diagram of different FRS models on CART based on Bonferroni-Dunn statistical method.Figure 6
CD diagram of different FRS models on LSVM based on Bonferroni-Dunn statistical method.


## 6. Conclusions

The development of robust FRS is a hot spot in the theory of FRS, which has some advantages in the feature selection of noise information. In this paper, the nonparametric fuzzy membership degree is defined by fuzzy granular calculation, and a FRS model based on representative samples is proposed. Firstly, the distance between the target sample and each representative sample is calculated by looking for the representative sample of each class, then the fuzzy membership degree of the target sample about all classes is determined, a FRS model based on the representative sample is proposed, and the relevant properties of the model are studied. On this basis, a feature selection algorithm based on FRS of representative samples is designed. Experimental results show that the RS-FRS model and feature selection algorithm proposed in this paper are feasible, effective, and robust for the processing of uncertain information systems with noise information, which expands the application field of FRS and the research on feature selection. However, the paper has some limitations. In this paper, the feature selection is carried out by the lower approximation of RS-FRS model. The lower approximation of the sample reflects the degree of certainty to which the sample belongs to the class, while the upper approximation of the sample reflects the degree of probability to which the sample belongs to the class. In the future research, it is considered to design feature selection algorithms using both upper approximation and lower approximation.



---

*Source: 6685396-2021-01-31.xml*