BACKGROUND
microarray techniques allow simultaneous measuring of the expression of thousands of genes under different experimental environments and conditions.
they allow us to analyse the gene information very rapidly by managing them at one time.
the gene expression profiles from particular microarray experiments have been widely used for cancer classification  <cit> .
however, the amount of data produced by this new technology is usually too large to be manually analysed.
hence, the need to automatically analyse the microarray data offers an opportunity for machine learning  methods to have a significant impact on cancer research.
the data from a series of m microarray experiments can be represented as an m × n gene expression matrix , where each row represents a sample described by the expression of n genes from one experiment.
each sample belongs to a certain class, i.e., cancer or non-cancer.
compared to some other classical problems in machine learning, microarray data sets pose various problems.
the number of features , usually in the range of  <dig> - <dig> , is much larger than the number of examples .
in addition, microarray data often brings in multiple missing gene expression values and noisy signals from the experiments.
therefore, classifying cancer mi-croarray gene expression data can be regarded as a high-dimensional-low-sample data problem with lots of noisy or missing data.
unsupervised methods, such as clustering  <cit> , and self-organizing maps   <cit>  were initially used to analyse the relationships among different genes.
subsequently, supervised methods, such as support vector machines   <cit> , multi-layer perceptrons   <cit> , k nearest neighbor  methods  <cit> , etc., have been successfully applied to the classification of different tissues.
but, most of the current methods in microarray data analysis are black box methods; these models can not satisfactorily reveal the hidden information in the data.
this information usually plays a very important role in making a quality clinical diagnosis.
different from black-box methods, fuzzy rule-based models can not only provide good classification results, but also easily be explained and interpreted in human understandable terms by using fuzzy rules.
this provides the researchers or clinician an insight into the developed models.
at the same time, fuzzy systems adapt numerical data  onto human linguistic terms and offer very good capabilities of dealing with noisy and missing data.
compared to other popular rule-based models in the area, such as c <dig>  decision trees   <cit> , the linguistic rules generated by our fuzzy-based models are short and easy to be read.
unfortunately, rule-based methods have suffered some well-known limitations in dealing with high dimensional data.
very high dimensional feature vectors and lack of enough training samples are two major challenges for modeling microarray data in general, hence for the success of applying fuzzy models to this problem too.
however, some recent developments in fuzzy systems provide us with some good ways to obtain good diagnosis results.
for example, vinterbo et al.
<cit>  firstly used fuzzy rule bases to classify gene expression data, but this model only allow linear discrimination, and the classification performance is limited; an adaptive-network-based fuzzy inference system  was successfully applied for this problem in  <cit>  too; woolf and wang  <cit>  developed a fuzzy based model to analyse the relationships between genes, while ressom et al.
<cit>  used a clustering-based preprocessing method to increase the efficiency of the fuzzy models.
all these reported systems are either small models which perform well on small data sets, or huge models which are difficult to be understood by the human experts.
other machine learning techniques, like genetic algorithms   <cit>  or ensemble learning  <cit>  have been adopted to allow fuzzy rule-based models to deal with a relative large number of features, but the obtained rule bases still look very complex to be easily comprehensible.
large model complexity significantly damages the main advantage of applying fuzzy models to this application, i.e., the inter-pretability of the models.
the computational cost of constructing these models is generally very high too.
normally, the accuracy of each fuzzy rule-based classifier is measured by the number of correctly classified training or testing patterns, while its in-terpretability is measured by the complexity of the model, more specifically, the number of fuzzy rules and the total number of antecedent conditions.
whereas both accuracy and interpretability were considered, multi-objective evolutionary based methods are introduced into our systems, hence, the name of multi-objective evolutionary algorithms based interpretable fuzzy  models.
we evaluated our proposed model on some well-known cancer data sets, i.e., the ovarian cancer data set, the lung cancer data set and the colon cancer data set.
experimental results are listed and discussed in the later section.
compared with most previously reported models, accurate and small fuzzy rule bases were obtained.
methods
gene selection
a major goal for diagnostic research is to develop diagnostic procedures based on inexpensive microarrays that have enough probes to detect certain diseases.
this requires the selection of some genes which are highly related to the particular classification problem, i.e., the informative genes.
this process is called gene selection , which corresponds to feature selection from any machine learning task in general.
two basic approaches for feature selection used in machine learning and information theory literature are the filter methods and the wrapper methods  <cit> .
in theory, wrapper methods should provide more accurate classification results than filter methods  <cit> .
the main disadvantage of the wrapper approach is its computational cost when combined with more complex algorithms such as svm for example.
the wrapper approach, which is popular in many machine learning applications, is not extensively used in dna microarray tasks, and in most cases the gene selection is performed by ranking genes on the basis of scores, correlation coefficients, mutual information and sensitivity analysis.
more detailed discussions of these two approaches can be found in  <cit> .
as suggested in  <cit> , a fuzzy c-mean clustering based enhanced gene selection method  is applied in this paper as well for gene selection.
improved methods for obtaining interpretable fuzzy models
an ideal design of fuzzy rule-based models for microarray data analysis is when we find fuzzy rule-based models with good interpretability but with acceptable testing accuracy too.
compared to most popular methods in cancer microarray gene expression data analysis area, rule-based fuzzy models usually have relative high computational complexity.
in order to obtain good fuzzy rule-based models, we adopted the following recent techniques to reduce the complexity of fuzzy rule-based models.
weighted fuzzy rules
the fuzzy rules rq used in our models are in the form of:
• rq: if x <dig> is aq <dig> and … and xn is aqn, then class cq with cfq.
in the above rule, x =  is the n-dimensional input vector, aqi is an antecedent fuzzy set for the i – th input variable, cq is a consequent class, and cfq is a certainty degree .
the rule weight is a real number in the unit interval  <cit> .
multiple fuzzy partitions
for a high-dimensional problem like microarray data analysis, the antecedent conditions of the generated rules are normally very numerous.
short fuzzy if-then rules with only a few antecedent conditions are obviously easy to understand for human users, and therefore a novel technique has been applied, as explained below.
we simultaneously generated  <dig> fuzzy sets from multiple fuzzy partitions, as shown in figure 1; “s”, “ms”, “m”, “ml” and “l” denote small, medium small , medium, medium large  and large, respectively.
the “don’t care”  condition has been added as an additional set.
there are  <dig> new fuzzy sets in total, and all of these fuzzy sets are fixed, without any tuning mechanism during the training.
after training, some fuzzy if-then rules may have n antecedent conditions , and others may have only a few antecedent conditions .
simple fuzzy reasoning
since we have  <dig> antecedent fuzzy sets for each attribute of our n-dimensional pattern classification problem, the total number of combinations of the antecedent fuzzy sets is 15n.
each combination is used as the antecedent part aq of the fuzzy rule rq.
its consequent class cq and rule weight cfqare specified from compatible training patterns with aq in the following heuristic manner.
first, we calculate the compatibility degree of each pattern xp with the antecedent part aq of the rule rq via a product operation like:
µaq = µaq <dig> ⋅…⋅µaqn
where µaq is the membership function of aqi.
then, the confidence of the fuzzy rule aq ⇒ class h is calculated for each class h as follows:
where m denotes the number of training patterns.
the consequent class cq is specified by identifying the class with the maximum confidence:
where m is the number of classes.
when there is no pattern in the fuzzy sub-space defined by aq, we do not generate any fuzzy rules with aq in the antecedent part.
this specification method of the consequent class of fuzzy rules has been used in a few studies since early 1990s  <cit> .
it should be noted that the same consequent class as in equations  <dig> and  <dig> is obtained when we use the support of the fuzzy rule aq ⇒ class h instead of the confidence degree.
the support of the fuzzy rule is calculated as follows:
different specifications for the rule weight cfq have been proposed and examined.
in this paper, we only consider binary classification problem, we can use the following specification because good results were previously reported for that in these papers  <cit> :
let s be a subset of candidate fuzzy rules, i.e., a fuzzy rule-based classifier.
each pattern xp is classified by a single winner rule rw, which is chosen from the rule set s as follows:
µaw ⋅ cfw = max{µaq ⋅ cfq|rq ∈ s}.
we only generate short fuzzy rules with a few antecedent conditions, and it should be noted that the dc conditions can be omitted from fuzzy rules.
this restriction is used in order to find a compact set of fuzzy rules with high interpretability.
as for short fuzzy rules, we only use fuzzy rules that satisfy both the minimum confidence and support as candidate rules for the multi-objective genetic fuzzy rule selection mechanism.
rule confidence and support can be used as pre-screening criteria for finding a tractable number of candidate fuzzy rules.
the generated fuzzy if-then rules are then divided into t groups according to their consequent classes, where t is a user-defined parameter.
fuzzy if-then rules in each group are sorted in the descending order of a pre-screening criterion .
for selecting q candidate rules, the first q/t rules are chosen from each of the t groups, and in this manner, we can choose an arbitrarily specified number of candidate fuzzy if-then rules .
it should be noted that the aim of the candidate rule pre-screening is not to construct a fuzzy rule-based system, but to find candidate rules, from which a small number of fuzzy if-then rules are later selected.
for using a variety of candidate rules in rule selection, we choose the same number of fuzzy if-then rules  for each class.
by applying these new techniques, the models complexity and computational cost is significantly decreased.
the multi-objective evolutionary algorithms based interpretable fuzzy  model
in this section, we describe how to apply multi-objective evolutionary algorithms  to extract fuzzy rule sets considering the balance between model accuracy and model interpretability.
our task is to select a smaller number of simple fuzzy if-then rules with high classification performance, and this is performed by maximizing the classification accuracy, minimizing the number of selected rules, and minimizing the total rule length at the same time.
therefore, the fitness value of each string s  in the current population is defined by the three objectives using the following fitness function:
f = w1⋅nccp –w2⋅nor –w3⋅noa,
where w = , nccp, nor, and noa are the weight vector, the number of correctly classified training patterns, the number of selected fuzzy rules in s, and the total number of antecedent conditions in s, respectively.
the weights w <dig> w <dig> w <dig> must satisfy the following conditions:
w <dig> w <dig> w3≥0;
w1+w2+w <dig> = 1;
as suggested by  <cit> , a rule subset r consisting of the q candidate rules can be represented by a binary string as:
r = r1r2r3…rq,
where rq =  <dig> means that the q-th candidate rule rq is not included in the rule set r, while rq' =  <dig> means that rq' is included in r.
to keep the model complexity low, a simple two-points crossover strategy is applied to each pair of parent strings to generate a new string.
biased mutation is applied to the generated string to efficiently decrease the number of fuzzy if-then rules included in each string, that is, different mutation probabilities are used for the mutation from  <dig> to  <dig> and the one from  <dig> to  <dig>  for example, the mutation probability from  <dig> to  <dig>  , and that from  <dig> to  <dig>  , are defined as:
where w <dig> w <dig> w <dig> are the user defined weights of the three search objectives in the fitness function, and pa <dig> and pa <dig> are two initial fixed parameters.
a larger probability is normally assigned to the mutation from  <dig> to  <dig> than to that from  <dig> to  <dig>  in order to efficiently decrease the number of fuzzy if-then rules  included in each string.
by applying the equations  <dig> and  <dig>  the mutation rate can be automatically adjusted according to user different purposes.
our moeaif can be summarized as follows:
<dig>  step 1: randomly generate npop  binary strings of length q as an initial population.
specify the crossover probability pc, two mutation probabilities, pa <dig> and pa <dig>  and the stopping condition;
<dig>  step 2: generate npop children strings by applying crossover and mutation to the current population;
<dig>  step 3: calculate the three-objectives fitness value for each string; unnecessary rules are removed from each string;
<dig>  step 4: update the next population by selecting top ranked individuals;
<dig>  step 5: stop, if the stopping condition is satisfied or the maximum number of training epochs is reached, otherwise return to step  <dig>
RESULTS
cancer data sets
we evaluated our proposed moeaif models on three cancer data sets, namely the ovarian cancer data set, the lung cancer data set and the colon cancer data set.
• lung cancer data set
lung cancer classification differentiates between malignant pleural mesothe-lioma  and adenocarcinoma  of the lung.
there are  <dig> reported samples in total, where  <dig> samples belong to mpm and  <dig> samples belong to adca.
the training set contains  <dig> samples,  <dig> mpm and  <dig> adca, and the remaining  <dig> samples are used for testing.
the expression levels of  <dig>  features were report in each sample.
each feature represents one probe, for example, the feature  <dig> at represents the probe  <dig> at.
the data is available at http://cilab.ujn.edu.cn/datasets.htm.
• ovarian cancer data set
the ovarian cancer data set was first reported in  <cit> .
the aim of the experiment was to identify proteomic patterns in serum that distinguish ovarian cancer from non-cancer.
this study is significant to women who have a high risk of ovarian cancer due to family or personal history of cancer.
the proteomic spectra were generated by mass spectroscopy and the raw data can be found at http://clinicalproteomics.steem.com.
there are  <dig> reported samples in this data set, where  <dig> samples belong to normal and  <dig> samples belong to ovarian cancers.
the normalization is done over all the  <dig> samples for all  <dig> m/z identities.
after the normalization, each intensity value is to fall within the range of  <dig> to  <dig>  the data is available at http://cilab.ujn.edu.cn/datasets.htm.
• colon cancer data set
the data set used here was firstly reported in  <cit> .
this data set contains  <dig> samples, of which  <dig> are tumour samples, and  <dig> normal samples.
about  <dig> genes are represented in each sample in the original data set, out of which only  <dig> were selected.
the data is available at http://sdmc.i2r.astar.edu.sg/rp/colontumor/colontumor.html.
accuracy of the moeaif models
different sets of w <dig>  w <dig> and w <dig> are used to simulate different users’ requirements.
from table  <dig> and table  <dig>  we can see that when classification accuracy is preferred, we can achieve the highest testing accuracy, at  <dig> % on lung cancer data, and  <dig> % on ovarian cancer data.
when the interpretability of the models is preferred, we can use two rules to classify lung cancer data with  <dig> % testing accuracy, and three rules to classify lung cancer data with a testing accuracy of  <dig> %.
very small rule bases for the lung cancer data set are obtained.
due to lack of enough training examples, a satisfactory testing accuracy on the colon data set was not obtained.
interpretability of the moeaif models
we further explore the rule base obtained by our moeaif approach for lung cancer data set, where w1= <dig> , w2= <dig> , and w3= <dig> .
the whole training process is given in figure  <dig>  the up left figure shows the fitness value of the best rule base found in the population during the training, the up right figure shows the testing accuracy given by the best rule base, the total number of fuzzy rules in the rule base is given in the lower left figure, and the total length of rules in the rule base is given in the lower right figure.
from these figures, a wide fluctuation of testing accuracy ) frequently appears at the early stage of training.
by analyzing the population, we notice this phenomenon means there must exist some very strong fuzzy rules which have large effect on the final classification result.
including these rules in the selected rule subset will significantly change the fitness value and classification accuracy during training.
in the training stage, this phenomenon can suggest to the users that a small, but useful fuzzy rule set exists in this data set.
a good testing accuracy is obtained within the first few generations, which shows that the pre-screening of candidate rules plays a very important role in this case.
multiple fuzzy partition technique also helps us avoid the high computational cost of adjusting fuzzy membership functions like some previously built fuzzy models.
compared to the lung cancer data set, it is more difficult to find an efficient small rule subset for the ovarian cancer data, see figure  <dig>  where the name of features are short for the names of m/z identities.
ovarian cancer data normally require a large number of genes, a large rule subset and a large number of initial candidate rules to obtain acceptable testing accuracy.
the algorithm can not converge when the number of the input genes is smaller than  <dig>  because the pre-processing stage has already defined a small search space, there is no need to set the number of generations to be a large number.
it would not help the algorithm to converge if some useful rules are not in the initial rule base, and the testing accuracy can also be difficult to be further improved during the training.
rule extraction
we can easily extract fuzzy rules from the lung cancer data set by using a trained moeaif model.
this fuzzy rule base can classify the testing data with the accuracy of  <dig>  by using only three rules .
four input features are used in the model, i.e., the feature  <dig> at, the feature  <dig> at, the feature  <dig> at and the feature  <dig> at.
from this table, we can see that the rules obtained by our moeaif model are linguistically interpretable, for example:
• rule 1: if the feature  <dig> at is “large” and the feature  <dig> at is “large”, then the sample is cancer with cf= <dig> %.
• rule 2: if the feature  <dig> at is “large” and the feature  <dig> at is “medium”, then the sample belongs to normal with cf =  <dig> %.
• rule 3: if the feature  <dig> at is “large” and the feature  <dig> at is “relatively small”, then the sample belongs to normal with cf =  <dig> %.
the membership functions of the feature  <dig> at and the feature  <dig> at in rule <dig> are “dont’t care”, which can reduce the length of rule <dig>  the rules generated by our moeaif models are shorter than the rules from our previously built models  <cit>  and some other reported rule-based models  <cit> .
CONCLUSIONS
in this paper, small and linguistically understandable fuzzy rule bases were obtained from challenging high dimensional cancer data sets by using our proposed multi-objective evolutionary algorithms based interpretable fuzzy  method.
the classification performance obtained by our models is also competitive.
we also point out that an ideal design of fuzzy rule-based models for microarray gene expression data analysis includes two important tasks: designing low-complexity fuzzy models, and finding trade-off points between classification accuracy and model interpretability.
we believe that fuzzy techniques and, in particular, the methods proposed in this paper can be very useful tools in dealing with microarray data.
there also are some important issues that need to be addressed in the future.
for example, some microarray gene expression data sets were generated directly from the probes set, and in some cases several probes may correspond to the same gene, or several different genes may hybridise to the same probes .
if some of the input features  in a single rule are specific to the same gene, then this rule need to be deleted.
due to lack of enough training examples, satisfactory classification results were not always guaranteed in some small data sets, for example, the colon cancer data set in this paper.
competing interests
the authors declare that they have no competing interests.
authors contributions
zhenyu wang has done the experiments and drafted the text, while vasile palade has contributed in guiding the experiments and checking and improving the text of the paper.

 microarray techniques allow simultaneous measuring of the expression of thousands of genes under different experimental environments and conditions. they allow us to analyse the gene information very rapidly by managing them at one time. the gene expression profiles from particular microarray experiments have been widely used for cancer classification. however, the amount of data produced by this new technology is usually too large to be manually analysed. hence, the need to automatically analyse the microarray data offers an opportunity for machine learning methods to have a significant impact on cancer research. the data from a series of m microarray experiments can be represented as an m × n gene expression matrix, where each row represents a sample described by the expression of n genes from one experiment. each sample belongs to a certain class, i.e., cancer or non-cancer. compared to some other classical problems in machine learning, microarray data sets pose various problems. the number of features, usually in the range of -, is much larger than the number of examples. in addition, microarray data often brings in multiple missing gene expression values and noisy signals from the experiments. therefore, classifying cancer mi-croarray gene expression data can be regarded as a high-dimensional-low-sample data problem with lots of noisy or missing data. unsupervised methods, such as clustering, and self-organizing maps were initially used to analyse the relationships among different genes. subsequently, supervised methods, such as support vector machines, multi-layer perceptrons, k nearest neighbor methods, etc., have been successfully applied to the classification of different tissues. but, most of the current methods in microarray data analysis are black box methods; these models can not satisfactorily reveal the hidden information in the data. this information usually plays a very important role in making a quality clinical diagnosis. different from black-box methods, fuzzy rule-based models can not only provide good classification results, but also easily be explained and interpreted in human understandable terms by using fuzzy rules. this provides the researchers or clinician an insight into the developed models. at the same time, fuzzy systems adapt numerical data onto human linguistic terms and offer very good capabilities of dealing with noisy and missing data. compared to other popular rule-based models in the area, such as c decision trees, the linguistic rules generated by our fuzzy-based models are short and easy to be read. unfortunately, rule-based methods have suffered some well-known limitations in dealing with high dimensional data. very high dimensional feature vectors and lack of enough training samples are two major challenges for modeling microarray data in general, hence for the success of applying fuzzy models to this problem too. however, some recent developments in fuzzy systems provide us with some good ways to obtain good diagnosis results. for example, vinterbo et al. firstly used fuzzy rule bases to classify gene expression data, but this model only allow linear discrimination, and the classification performance is limited; an adaptive-network-based fuzzy inference system was successfully applied for this problem in too; woolf and wang developed a fuzzy based model to analyse the relationships between genes, while ressom et al. used a clustering-based preprocessing method to increase the efficiency of the fuzzy models. all these reported systems are either small models which perform well on small data sets, or huge models which are difficult to be understood by the human experts. other machine learning techniques, like genetic algorithms or ensemble learning have been adopted to allow fuzzy rule-based models to deal with a relative large number of features, but the obtained rule bases still look very complex to be easily comprehensible. large model complexity significantly damages the main advantage of applying fuzzy models to this application, i.e., the inter-pretability of the models. the computational cost of constructing these models is generally very high too. normally, the accuracy of each fuzzy rule-based classifier is measured by the number of correctly classified training or testing patterns, while its in-terpretability is measured by the complexity of the model, more specifically, the number of fuzzy rules and the total number of antecedent conditions. whereas both accuracy and interpretability were considered, multi-objective evolutionary based methods are introduced into our systems, hence, the name of multi-objective evolutionary algorithms based interpretable fuzzy models. we evaluated our proposed model on some well-known cancer data sets, i.e., the ovarian cancer data set, the lung cancer data set and the colon cancer data set. experimental results are listed and discussed in the later section. compared with most previously reported models, accurate and small fuzzy rule bases were obtained. methods gene selection a major goal for diagnostic research is to develop diagnostic procedures based on inexpensive microarrays that have enough probes to detect certain diseases. this requires the selection of some genes which are highly related to the particular classification problem, i.e., the informative genes. this process is called gene selection, which corresponds to feature selection from any machine learning task in general. two basic approaches for feature selection used in machine learning and information theory literature are the filter methods and the wrapper methods. in theory, wrapper methods should provide more accurate classification results than filter methods. the main disadvantage of the wrapper approach is its computational cost when combined with more complex algorithms such as svm for example. the wrapper approach, which is popular in many machine learning applications, is not extensively used in dna microarray tasks, and in most cases the gene selection is performed by ranking genes on the basis of scores, correlation coefficients, mutual information and sensitivity analysis. more detailed discussions of these two approaches can be found in. as suggested in, a fuzzy c-mean clustering based enhanced gene selection method is applied in this paper as well for gene selection. improved methods for obtaining interpretable fuzzy models an ideal design of fuzzy rule-based models for microarray data analysis is when we find fuzzy rule-based models with good interpretability but with acceptable testing accuracy too. compared to most popular methods in cancer microarray gene expression data analysis area, rule-based fuzzy models usually have relative high computational complexity. in order to obtain good fuzzy rule-based models, we adopted the following recent techniques to reduce the complexity of fuzzy rule-based models. weighted fuzzy rules the fuzzy rules rq used in our models are in the form of: • rq: if x is aq and … and xn is aqn, then class cq with cfq. in the above rule, x = is the n-dimensional input vector, aqi is an antecedent fuzzy set for the i – th input variable, cq is a consequent class, and cfq is a certainty degree. the rule weight is a real number in the unit interval. multiple fuzzy partitions for a high-dimensional problem like microarray data analysis, the antecedent conditions of the generated rules are normally very numerous. short fuzzy if-then rules with only a few antecedent conditions are obviously easy to understand for human users, and therefore a novel technique has been applied, as explained below. we simultaneously generated fuzzy sets from multiple fuzzy partitions, as shown in figure 1; “s”, “ms”, “m”, “ml” and “l” denote small, medium small, medium, medium large and large, respectively. the “don’t care” condition has been added as an additional set. there are new fuzzy sets in total, and all of these fuzzy sets are fixed, without any tuning mechanism during the training. after training, some fuzzy if-then rules may have n antecedent conditions, and others may have only a few antecedent conditions. simple fuzzy reasoning since we have antecedent fuzzy sets for each attribute of our n-dimensional pattern classification problem, the total number of combinations of the antecedent fuzzy sets is 15n. each combination is used as the antecedent part aq of the fuzzy rule rq. its consequent class cq and rule weight cfqare specified from compatible training patterns with aq in the following heuristic manner. first, we calculate the compatibility degree of each pattern xp with the antecedent part aq of the rule rq via a product operation like: µaq = µaq ⋅…⋅µaqn where µaq is the membership function of aqi. then, the confidence of the fuzzy rule aq ⇒ class h is calculated for each class h as follows: where m denotes the number of training patterns. the consequent class cq is specified by identifying the class with the maximum confidence: where m is the number of classes. when there is no pattern in the fuzzy sub-space defined by aq, we do not generate any fuzzy rules with aq in the antecedent part. this specification method of the consequent class of fuzzy rules has been used in a few studies since early 1990s. it should be noted that the same consequent class as in equations and is obtained when we use the support of the fuzzy rule aq ⇒ class h instead of the confidence degree. the support of the fuzzy rule is calculated as follows: different specifications for the rule weight cfq have been proposed and examined. in this paper, we only consider binary classification problem, we can use the following specification because good results were previously reported for that in these papers: let s be a subset of candidate fuzzy rules, i.e., a fuzzy rule-based classifier. each pattern xp is classified by a single winner rule rw, which is chosen from the rule set s as follows: µaw ⋅ cfw = max{µaq ⋅ cfq|rq ∈ s}. we only generate short fuzzy rules with a few antecedent conditions, and it should be noted that the dc conditions can be omitted from fuzzy rules. this restriction is used in order to find a compact set of fuzzy rules with high interpretability. as for short fuzzy rules, we only use fuzzy rules that satisfy both the minimum confidence and support as candidate rules for the multi-objective genetic fuzzy rule selection mechanism. rule confidence and support can be used as pre-screening criteria for finding a tractable number of candidate fuzzy rules. the generated fuzzy if-then rules are then divided into t groups according to their consequent classes, where t is a user-defined parameter. fuzzy if-then rules in each group are sorted in the descending order of a pre-screening criterion. for selecting q candidate rules, the first q/t rules are chosen from each of the t groups, and in this manner, we can choose an arbitrarily specified number of candidate fuzzy if-then rules. it should be noted that the aim of the candidate rule pre-screening is not to construct a fuzzy rule-based system, but to find candidate rules, from which a small number of fuzzy if-then rules are later selected. for using a variety of candidate rules in rule selection, we choose the same number of fuzzy if-then rules for each class. by applying these new techniques, the models complexity and computational cost is significantly decreased. the multi-objective evolutionary algorithms based interpretable fuzzy model in this section, we describe how to apply multi-objective evolutionary algorithms to extract fuzzy rule sets considering the balance between model accuracy and model interpretability. our task is to select a smaller number of simple fuzzy if-then rules with high classification performance, and this is performed by maximizing the classification accuracy, minimizing the number of selected rules, and minimizing the total rule length at the same time. therefore, the fitness value of each string s in the current population is defined by the three objectives using the following fitness function: f = w1⋅nccp –w2⋅nor –w3⋅noa, where w =, nccp, nor, and noa are the weight vector, the number of correctly classified training patterns, the number of selected fuzzy rules in s, and the total number of antecedent conditions in s, respectively. the weights w w w must satisfy the following conditions: w w w3≥0; w1+w2+w = 1; as suggested by, a rule subset r consisting of the q candidate rules can be represented by a binary string as: r = r1r2r3…rq, where rq = means that the q-th candidate rule rq is not included in the rule set r, while rq' = means that rq' is included in r. to keep the model complexity low, a simple two-points crossover strategy is applied to each pair of parent strings to generate a new string. biased mutation is applied to the generated string to efficiently decrease the number of fuzzy if-then rules included in each string, that is, different mutation probabilities are used for the mutation from to and the one from to for example, the mutation probability from to, and that from to, are defined as: where w w w are the user defined weights of the three search objectives in the fitness function, and pa and pa are two initial fixed parameters. a larger probability is normally assigned to the mutation from to than to that from to in order to efficiently decrease the number of fuzzy if-then rules included in each string. by applying the equations and the mutation rate can be automatically adjusted according to user different purposes. our moeaif can be summarized as follows: step 1: randomly generate npop binary strings of length q as an initial population. specify the crossover probability pc, two mutation probabilities, pa and pa and the stopping condition; step 2: generate npop children strings by applying crossover and mutation to the current population; step 3: calculate the three-objectives fitness value for each string; unnecessary rules are removed from each string; step 4: update the next population by selecting top ranked individuals; step 5: stop, if the stopping condition is satisfied or the maximum number of training epochs is reached, otherwise return to step  cancer data sets we evaluated our proposed moeaif models on three cancer data sets, namely the ovarian cancer data set, the lung cancer data set and the colon cancer data set. • lung cancer data set lung cancer classification differentiates between malignant pleural mesothe-lioma and adenocarcinoma of the lung. there are reported samples in total, where samples belong to mpm and samples belong to adca. the training set contains samples, mpm and adca, and the remaining samples are used for testing. the expression levels of features were report in each sample. each feature represents one probe, for example, the feature at represents the probe at. the data is available at http://cilab.ujn.edu.cn/datasets.htm. • ovarian cancer data set the ovarian cancer data set was first reported in. the aim of the experiment was to identify proteomic patterns in serum that distinguish ovarian cancer from non-cancer. this study is significant to women who have a high risk of ovarian cancer due to family or personal history of cancer. the proteomic spectra were generated by mass spectroscopy and the raw data can be found at http://clinicalproteomics.steem.com. there are reported samples in this data set, where samples belong to normal and samples belong to ovarian cancers. the normalization is done over all the samples for all m/z identities. after the normalization, each intensity value is to fall within the range of to the data is available at http://cilab.ujn.edu.cn/datasets.htm. • colon cancer data set the data set used here was firstly reported in. this data set contains samples, of which are tumour samples, and normal samples. about genes are represented in each sample in the original data set, out of which only were selected. the data is available at http://sdmc.i2r.astar.edu.sg/rp/colontumor/colontumor.html. accuracy of the moeaif models different sets of w w and w are used to simulate different users’ requirements. from table and table we can see that when classification accuracy is preferred, we can achieve the highest testing accuracy, at % on lung cancer data, and % on ovarian cancer data. when the interpretability of the models is preferred, we can use two rules to classify lung cancer data with % testing accuracy, and three rules to classify lung cancer data with a testing accuracy of %. very small rule bases for the lung cancer data set are obtained. due to lack of enough training examples, a satisfactory testing accuracy on the colon data set was not obtained. interpretability of the moeaif models we further explore the rule base obtained by our moeaif approach for lung cancer data set, where w1=, w2=, and w3=. the whole training process is given in figure the up left figure shows the fitness value of the best rule base found in the population during the training, the up right figure shows the testing accuracy given by the best rule base, the total number of fuzzy rules in the rule base is given in the lower left figure, and the total length of rules in the rule base is given in the lower right figure. from these figures, a wide fluctuation of testing accuracy ) frequently appears at the early stage of training. by analyzing the population, we notice this phenomenon means there must exist some very strong fuzzy rules which have large effect on the final classification result. including these rules in the selected rule subset will significantly change the fitness value and classification accuracy during training. in the training stage, this phenomenon can suggest to the users that a small, but useful fuzzy rule set exists in this data set. a good testing accuracy is obtained within the first few generations, which shows that the pre-screening of candidate rules plays a very important role in this case. multiple fuzzy partition technique also helps us avoid the high computational cost of adjusting fuzzy membership functions like some previously built fuzzy models. compared to the lung cancer data set, it is more difficult to find an efficient small rule subset for the ovarian cancer data, see figure where the name of features are short for the names of m/z identities. ovarian cancer data normally require a large number of genes, a large rule subset and a large number of initial candidate rules to obtain acceptable testing accuracy. the algorithm can not converge when the number of the input genes is smaller than because the pre-processing stage has already defined a small search space, there is no need to set the number of generations to be a large number. it would not help the algorithm to converge if some useful rules are not in the initial rule base, and the testing accuracy can also be difficult to be further improved during the training. rule extraction we can easily extract fuzzy rules from the lung cancer data set by using a trained moeaif model. this fuzzy rule base can classify the testing data with the accuracy of by using only three rules. four input features are used in the model, i.e., the feature at, the feature at, the feature at and the feature at. from this table, we can see that the rules obtained by our moeaif model are linguistically interpretable, for example: • rule 1: if the feature at is “large” and the feature at is “large”, then the sample is cancer with cf= %. • rule 2: if the feature at is “large” and the feature at is “medium”, then the sample belongs to normal with cf = %. • rule 3: if the feature at is “large” and the feature at is “relatively small”, then the sample belongs to normal with cf = %. the membership functions of the feature at and the feature at in rule are “dont’t care”, which can reduce the length of rule the rules generated by our moeaif models are shorter than the rules from our previously built models and some other reported rule-based models.  in this paper, small and linguistically understandable fuzzy rule bases were obtained from challenging high dimensional cancer data sets by using our proposed multi-objective evolutionary algorithms based interpretable fuzzy method. the classification performance obtained by our models is also competitive. we also point out that an ideal design of fuzzy rule-based models for microarray gene expression data analysis includes two important tasks: designing low-complexity fuzzy models, and finding trade-off points between classification accuracy and model interpretability. we believe that fuzzy techniques and, in particular, the methods proposed in this paper can be very useful tools in dealing with microarray data. there also are some important issues that need to be addressed in the future. for example, some microarray gene expression data sets were generated directly from the probes set, and in some cases several probes may correspond to the same gene, or several different genes may hybridise to the same probes. if some of the input features in a single rule are specific to the same gene, then this rule need to be deleted. due to lack of enough training examples, satisfactory classification results were not always guaranteed in some small data sets, for example, the colon cancer data set in this paper. competing interests the authors declare that they have no competing interests. authors contributions zhenyu wang has done the experiments and drafted the text, while vasile palade has contributed in guiding the experiments and checking and improving the text of the paper. 
different from most other popular methods in high dimensional bio-medical data analysis, such as microarray gene expression or proteomics mass spectroscopy data analysis, fuzzy rule-based models can not only provide good classification results, but also easily be explained and interpreted in human understandable terms, by using fuzzy rules. we mainly focus on evaluating our proposed models on microarray gene expression cancer data sets, i.e., the lung cancer data set and the colon cancer data set, but we extend our investigations to other type of cancer data set, such as the ovarian cancer data set. although some recently developed fuzzy-based modeling approaches can provide satisfactory classification results, the rule bases generated by most of the reported fuzzy models for gene expression data are still too large to be easily comprehensible. the experimental studies have shown that relatively simple and small fuzzy rule bases, with satisfactory classification performance, can be successfully obtained for challenging microarray gene expression datasets. we believe that fuzzy-based techniques, and in particular the methods proposed in this paper, can be very useful tools in dealing with high dimensional cancer data. however, the advantages offered by fuzzy-based techniques in microarray data analysis have not yet been fully explored in the literature. we also argue that the potential of applying fuzzy-based techniques to microarray data analysis need to be further explored. in this paper, we develop some multi-objective evolutionary algorithms based interpretable fuzzy methods for analysing high dimensional bio-medical data sets, such as microarray gene expression data and proteomics mass spectroscopy data.
 analysing gene expression data from microarray technologies is a very important task in biology and medicine, and particularly in cancer diagnosis. different from most other popular methods in high dimensional bio-medical data analysis, such as microarray gene expression or proteomics mass spectroscopy data analysis, fuzzy rule-based models can not only provide good classification results, but also easily be explained and interpreted in human understandable terms, by using fuzzy rules. however, the advantages offered by fuzzy-based techniques in microarray data analysis have not yet been fully explored in the literature. although some recently developed fuzzy-based modeling approaches can provide satisfactory classification results, the rule bases generated by most of the reported fuzzy models for gene expression data are still too large to be easily comprehensible.  in this paper, we develop some multi-objective evolutionary algorithms based interpretable fuzzy methods for analysing high dimensional bio-medical data sets, such as microarray gene expression data and proteomics mass spectroscopy data. we mainly focus on evaluating our proposed models on microarray gene expression cancer data sets, i.e., the lung cancer data set and the colon cancer data set, but we extend our investigations to other type of cancer data set, such as the ovarian cancer data set. the experimental studies have shown that relatively simple and small fuzzy rule bases, with satisfactory classification performance, can be successfully obtained for challenging microarray gene expression datasets.  we believe that fuzzy-based techniques, and in particular the methods proposed in this paper, can be very useful tools in dealing with high dimensional cancer data. we also argue that the potential of applying fuzzy-based techniques to microarray data analysis need to be further explored. 18- december ieee international conference on bioinformatics and biomedicine hong kong, p. r. china 