# Comparative Study of Embedded Feature Selection Methods on Microarray Data

Hind Hamla, Khadoudja Ghanem

<figure><img src="image_1.png" /></figure>

## To cite this version:

Hind Hamla, Khadoudja Ghanem. Comparative Study of Embedded Feature Selection Methods on Microarray Data. 17th IFIP International Conference on Artificial Intelligence Applications and Innovations (AIAI), Jun 2021, Hersonissos, Crete, Greece. pp.69-77, 10.1007/978-3-030-79150-6_6 hal-03287701

HAL Id: hal-03287701

https://inria.hal.science/hal-03287701

Submitted on 15 Jul 2021

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Comparative Study of Embedded Feature
Selection Methods on Microarray data

Hind Hamla¹[0000-0002-7062-7795] and Khadoudja Ghanem²

¹ University of Abdelhamid Mehri constantine2, Constantine, Algeria

² Laboratory of Modelling and Implementation of Complex Systems
hind.hamla@univ-constantine2.dz

³ University of Abdelhamid Mehri constantine2, Constantine, Algeria

⁴ Laboratory of Modelling and Implementation of Complex Systems
gkhadoudja@yahoo.fr

**Abstract.** Microarray data collects information from tissues that could be used in early diagnosis such as cancer. However, the classification of microarray data is a challenging task due to the high number of features and a small number of samples leading to poor classification accuracy. Feature selection is very effective in reducing dimensionality; it eliminates redundant and irrelevant features to enhance the classifier's performance. In order to shed light on the strengths and weaknesses of the existing techniques, we compare the performances of five embedded feature selection methods namely decision tree, random forest, lasso, ridge, and SVM-RFE. Ten well-known microarray datasets are tested. Obtained results show the outperformance of SVM-RFE in term of accuracy, and comes in the second position after decision tree in terms of number of selected features and execution time.

**Keywords:** Feature selection · Machine learning · Embedded methods · Microarray data · SVM-RFE.

# 1 Introduction

Artificial intelligence AI based learning is able to substantially "automate discovery" across many domains where classification and prediction tasks play an important role. It holds out the prospect of dramatically lower costs and improve performance especially with DNA microarray data. Within this specific field, AI based learning can learn to read the genome in ways that human cognition and perception cannot as it was argued by Leung et al 2016.[13]

DNA Microarray technology collects information from tissues and cell samples, analyze this type of data allows research to early diagnose diseases such as cancer [10]. Artificial intelligence techniques have made it possible to analyze this data for classification using statistical methods. However, there are often a few samples (often fewer than 100 samples) and a huge number of features in the raw data up to 60,000 which causes the curse of dimensionality [2]. According to [23] only a small number of features are used in classification, hence,
feature selection plays an important role in removing irrelevant features from
microarray data. Feature selection is used for dimensionality reduction, it elim-
inates irrelevant and redundant features and preserves all or most informative
features [22]. As mentioned in [20], feature selection improves the performance
of machine learning algorithms either by increasing the classification accuracy
or by decreasing the learning speed.

Feature selection methods are categorized into three categories namely: filter, wrapper, and embedded [4]. Wrapper methods use the classification model to measure the goodness of features [14], these methods are slower than filter methods but they have better performance. Filter methods, however, select features independently from classification models and use statistical proprieties of data to select features. These methods are more time-saving compared to wrapper methods, but they show poorer performance [14]. Embedded methods are a combination of filters and wrapper methods. They integrate the process of selection features in the model training phase. With these methods, the search for the best features subset is guided by the learning model. Embedded methods perform better than filter methods and have less computational time compared to wrapper methods. This is way we explore in this paper this category of feature selection methods.

In order to shed light on the strengths and weaknesses of the existing tech-
niques, we compare the performances of five embedded feature selection meth-
ods namely: decision tree, random forest, lasso, ridge, and SVM-RFE. This work
tests ten well-known microarray datasets that suffer from the high dimensional
problem. The remainder of the paper is organized as follows: Section 2 presents
the related works and Section 3 describes the five embedded methods used in
this study. Experimental results are given and discussed in Section 4. Finally,
Section 5 concludes the paper.

## 2 Related Works

Wrapper methods suffer from high computational cost while filter methods do not interact with classifiers [12]. Embedded methods can solve these two problems by including the feature selection method in the learning process. In [20], some of the popular feature selection methods were reviewed, and the performance of some of these methods was evaluated in medical domain. Authors in [23], presented a comprehensive state of the art of many existing feature selection methods. Below we present an overview of several recent embedded methods.

In [9], the authors proposed an embedded feature selection method named Re-cursive Feature Addition (RFA), RFA worked in a forward fashion and based on SVM. The authors used five datasets to test the performance of the proposed method, the results showed superior performance of the proposed method over filter, wrapper, and other embedded methods.

A stable feature selection method based on the L1-norm support vector machine (SVM) was proposed in [19], the proposed method combined L1-norm with SVM classifier for the classification of renal clear cell carcinoma stage classification us-
ing backward feature elimination. RNA-seq gene expression dataset was used to
evaluate the performance of the proposed method. The results showed the out-
performance of the proposed method.

The authors of [5] propose an embedded feature selection method named MAGS for gene selection and classification of microarray data. This method uses SVM not only to evaluate the quality of the subsets but also to provide valuable information about gene relevancy to design specialized crossover and local search operators. The performance of the proposed method was evaluated over eight microarray datasets, the results illustrate that the proposed method improves the classification accuracy.

In [16] the authors proposed an embedded feature selection method to solve the class imbalance problem called GI-FSw. In this approach, the weighted Gini index was used as splitting criteria of the CART decision tree classifier. The proposed method was tested over two datasets and it has achieved high levels of ROC and f-measure.

Markov blanket-embedded genetic algorithm (MBEGA) for gene selection prob-
lem was proposed in [24], in this approach, Markov blanket-based deleted or
added features from a solution of genetic algorithm in order to improve the final
solution, the performance of this approach was evaluated over four synthetic
datasets and eleven microarray datasets, the results proved the effectiveness of
the proposed method in eliminating redundant and irrelevant features and it
outperforms existing methods in terms of accuracy, number of selected features
and computational cost.

In [15], the authors proposed an improved version of RFE that used variable step size named (VSSRFE) in which the step size was decreased as the number of features is eliminated. This method was combined with a more efficient implementation of SVM called (LLSVM). Six well-known microarray datasets were used in the experiments, the results showed that the proposed methods have obtained a comparable classification performance and reduced the computational time.

From this overview, we found that most of the embedded methods presented
in the literature to solve feature selection problem are: Decision tree, Rondom
Forest, SVM associated with different techniques like L1, L2, RFE...Like it has
been mentioned. SVM-RFE was compared in [15] with new variants of SVM-
RFE and it has been shown that these variants outperform SVM-RFE. In this
paper, we compare DT, RF, L1, and L2 with SVM-RFE on specific data which
are microarray data.

## 3 Methods and materials

As stated before, in embedded methods the selection of features and the learning
are dependent. In the following section, the principles of the used embedded
methods are introduced.
## 3.1 Decision tree

Decision tree [21] is a well-known machine learning classifier in form of a tree structure that consists of decision nodes and leaves, whereas each leaf represents a class. Decision tree selects features. This classifier is known by its ability to select features that are important in classification. There are several algorithms for constructing a decision tree using different splitting criteria for example ID3 uses impurity measure, C4.5 uses the concept of information entropy, and CART uses Gini index [16]. Decision tree is able of selecting informative features required for classification because the selection of features is inherent in the decision tree classifier [7].

## 3.2 Random Forest

Random Forest was proposed by [3], it is an embedded feature selection method that uses variable importance to select features. Random forest constructs a large number of classification trees using bootstrapped samples and determines the predictive class using majority voting. This method is well suited for microarray data because it shows excellent performance when the number of samples is much less than the number of features [4].

## 3.3 Lasso

Least Absolute Shrinkage and Selection Operator (LASSO) is an embedded feature selection method proposed by Robert Tibshirani in 1996, it applies a regularization process that penalizes the coefficients of the regression variables (features) shrinking some of them to zero [6] and preserves features that still have non-zero coefficient after the regularization process. Lasso is an effective feature selection method considering high-dimensional data due to its ability to produce sparse models in a reasonable time [17].

## 3.4 Ridge

Ridge is another shrinkage-based embedded method proposed by (Hoerl and Kennard, 1988) [11], it penalizes the square root of the sum of the squared weights (L2 norm). This method applies the regularization process by penalizing the feature coefficients toward zero but never attempt exactly zero [18]. Ridge does not reduce the number of variables since it never leads to a zero coefficient.

## 3.5 SVM-RFE

SVM-RFE was introduced by [8], it is an iteration backward elimination method that trains SVM with the current set of features and removes the least performing features indicated with SVM recursively until finding the optimal subset [20]. SVM-RFE steps for feature set selection are shown as follows.
- Train the classifier (SVM) with the current dataset.

- Calculates the ranking criterion for all features.

- Delete the least important features (has the smallest ranking criterion).

More than one feature can be removed in one iteration [1].

# 4 Experimental Results and Discussion

## 4.1 Datasets

To analyze and test the embedded feature selection performance, ten well-known microarray datasets [24] for various cancer diagnoses were used. Table 1 shows datasets details. The above datasets are partitioned into training and testing sets,

<table><caption>Table 1. Microarray Datasets Details.</caption><thead><tr><td>Datasets</td><td>Number of features</td><td>Number of instances</td><td>Number of classes</td></tr></thead><tbody><tr><td>Colon tumor</td><td>2000</td><td>60</td><td>2</td></tr><tr><td>Central Nervous System</td><td>7129</td><td>60</td><td>2</td></tr><tr><td>Leukemia</td><td>7129</td><td>72</td><td>2</td></tr><tr><td>Breast cancer</td><td>24481</td><td>97</td><td>2</td></tr><tr><td>Lung cancer</td><td>12533</td><td>181</td><td>2</td></tr><tr><td>Ovarian cancer</td><td>15154</td><td>253</td><td>2</td></tr><tr><td>Leukemia 3 classes</td><td>7129</td><td>72</td><td>3</td></tr><tr><td>Lymphoma 4 classes</td><td>7129</td><td>72</td><td>4</td></tr><tr><td>Lymphoma</td><td>4026</td><td>62</td><td>3</td></tr><tr><td>MLL</td><td>12582</td><td>72</td><td>3</td></tr></tbody></table>

eight parts were used to train the classifiers and the remaining two were used to test the classifiers. We replace missing values with the mean of the observed values for the corresponding feature. The 10-fold cross-validation method was used in this study.

## 4.2 Experimental Setup

The five methods used in this study were implemented in python using scikit-learn library. All experiments are performed on a Personal Computer (PC) with an Intel Core i7 processor, 2.9 GHz, and 8 GB of RAM. The parameters values selected to perform the experiments are recorded in tables 2 and 3. It worth to mention that we perform several runs to select the suited parameters.
<table><caption>**Table 2.** List of classification algorithms with parameter tuning details.</caption><thead><tr><th>Classifiers Name</th><th>Parameter Tuning</th></tr></thead><tbody><tr><td>Decision tree (DT)</td><td>Criterion = 'gini', splitter = 'best', minSamples_split= 2<br>minSamples_leaf= 1, presort= 'deprecated'</td></tr><tr><td>Random Forest (RF)</td><td>nEstimators= 100, criterion= 'gini', minSamples_split= 2<br>minSamples_leaf= 1, bootstrap = True</td></tr><tr><td>Lasso</td><td>dual= false, tol= 1e-4<br>C= 1.0, fitIntercept= True</td></tr><tr><td>Ridge</td><td>Alpha = 1.0, fitIntercept= True<br>normalize= False, tol= 1e-3</td></tr><tr><td>SVM-RFE</td><td>C =1.0, Kernel= 'linear', degree= 3, gamma= 'scale'<br>shrinking= True, tol= 1e-3, step= 1</td></tr></tbody></table>

In addition, we set the max-depth parameter of decision tree classifier max-depth = 10 for all the data sets except for colon and leukemia-4C max-depth=20 and breast max-depth =100.

Table 3 presents the classification accuracy and execution time in milliseconds without using any feature selection methods, whereas, table 4 presents classification accuracy and execution time (in millisecond) and the number of selected features using embedded methods. The results are the average of five runs.

## 4.3 Discussion

From the tables above it is evident that using embedded methods significantly improves classification accuracy and reduces execution time. SVM-RFE outperforms all methods on the ten used datasets in term of accuracy using less than twenty features, it is worth to mention that SVM-RFE reaches 100% accuracy in the case of seven datasets, and it comes in the second place after decision tree in term of execution time and the number of selected features. Decision tree selects the least number of features and has the best execution time on nine of the ten datasets, it also has the best performance in term of accuracy after SVM-RFE in the case of five datasets (colon, CNS, leukemia 4-C, breast, and lung), with regard to the other datasets, the accuracy remains quite important. Random forest has the lowest classification accuracy and requires high execution time, however, the number of selected features is reasonable. Thus, when considering these three parameters (accuracy, Execution time and number of selected features), it is suggested to avoid this method in the presence of this type of datasets. But, lasso has average results when compared to the other methods and ridge provides sufficient classification accuracy however it requires the longest execution time on almost all the datasets and it selects the largest number of features (over one thousand features for nine datasets). Finally, we think that it remains preferable to leave the choice to the user, who will use the method that suits his needs in terms of precision (very Sensitive applications), time (Real time applications) or number of features (Mobile applications) ...
<table><caption>Table 3: Classification accuracy and execution time using all features.</caption><thead><tr><th>Datasets</th><th></th><th>DT</th><th>RF</th><th>Lasso</th><th>Ridge</th><th>RFE-SVM</th></tr></thead><tbody><tr><td rowspan="2">Colon</td><td>Time</td><td>16.18</td><td>10.60</td><td>10.20</td><td>7.32</td><td>8414.06</td></tr><tr><td>ACC</td><td>75.46%</td><td>75.63%</td><td>85.76%</td><td>78.16%</td><td>75.16%</td></tr><tr><td rowspan="2">CNS</td><td>Time</td><td>71.03</td><td>13.86</td><td>49.57</td><td>8.82</td><td>181601.87</td></tr><tr><td>ACC</td><td>44.66%</td><td>62.83%</td><td>70.83%</td><td>72.5%</td><td>72.0%</td></tr><tr><td rowspan="2">Leukemia</td><td>Time</td><td>73.31</td><td>41.11</td><td>88.24</td><td>82.25</td><td>169159.77</td></tr><tr><td>ACC</td><td>87.44%</td><td>91.10%</td><td>97.71%</td><td>96.33%</td><td>93.14%</td></tr><tr><td rowspan="2">Leukemia 3-C</td><td>Time</td><td>85.84</td><td>39.79</td><td>168.67</td><td>77.07</td><td>159072.89</td></tr><tr><td>ACC</td><td>84.25%</td><td>84.62%</td><td>96.57%</td><td>94.57%</td><td>96.0%</td></tr><tr><td rowspan="2">Leukemia 4-C</td><td>Time</td><td>131.84</td><td>93.22</td><td>326.56</td><td>95.02</td><td>255629.21</td></tr><tr><td>ACC</td><td>69.90%</td><td>77.33%</td><td>92.92%</td><td>91.21%</td><td>94.64%</td></tr><tr><td rowspan="2">Breast</td><td>Time</td><td>443.38</td><td>34.45</td><td>182.76</td><td>35.85</td><td>4758377.28</td></tr><tr><td>ACC</td><td>52.49%</td><td>61.05%</td><td>72.43%</td><td>66.58%</td><td>64.44%</td></tr><tr><td rowspan="2">Lung cancer</td><td>Time</td><td>673.86</td><td>63.49</td><td>965.80</td><td>49.48</td><td>2016552.5</td></tr><tr><td>ACC</td><td>85.40%</td><td>88.55%</td><td>92.63%</td><td>93.28%</td><td>90.42</td></tr><tr><td rowspan="2">Ovarian</td><td>Time</td><td>639.34</td><td>225.18</td><td>352.96</td><td>435.79</td><td>2099057.88</td></tr><tr><td>ACC</td><td>96.27%</td><td>97.09%</td><td>100.0%</td><td>100.0%</td><td>100.0%</td></tr><tr><td rowspan="2">Lymphoma</td><td>Time</td><td>62.29</td><td>50.37</td><td>79.70</td><td>107.13</td><td>37563.19</td></tr><tr><td>ACC</td><td>85.1%</td><td>93.32%</td><td>94.16%</td><td>100.0%</td><td>96.66%</td></tr><tr><td rowspan="2">MLL</td><td>Time</td><td>147.90</td><td>51.48</td><td>291.64</td><td>115.85</td><td>444714.36</td></tr><tr><td>ACC</td><td>80.54%</td><td>90.95%</td><td>98.57%</td><td>96.07%</td><td>90.23%</td></tr></tbody></table>

<table><caption>Table 4: Classification accuracy and execution time and the number of selected features.</caption><thead><tr><th>Datasets</th><th></th><th>DT</th><th>RF</th><th>L1</th><th>L2</th><th>SVM-RFE</th></tr></thead><tbody><tr><td rowspan="2">ColonTumor</td><td>Time</td><td>0.55</td><td>12.74</td><td>4.85</td><td>3.45</td><td>1.33</td></tr><tr><td>ACC</td><td>91.9%</td><td>80.56%</td><td>84.16%</td><td>89.49%</td><td>96.0%</td></tr><tr><td></td><td>Nb FS</td><td>4</td><td>42</td><td>204.6</td><td>590</td><td>10</td></tr><tr><td rowspan="2">CNS</td><td>Time</td><td>0.34</td><td>9.43</td><td>2.07</td><td>2.77</td><td>1.09</td></tr><tr><td>ACC</td><td>89.89%</td><td>76.09%</td><td>79.33%</td><td>81.0%</td><td>98.00%</td></tr><tr><td></td><td>Nb FS</td><td>4</td><td>44.2</td><td>290.6</td><td>1822</td><td>15</td></tr><tr><td rowspan="2">Leukemia</td><td>Time</td><td>0.79</td><td>9.72</td><td>3.08</td><td>12.96</td><td>1.31</td></tr><tr><td>ACC</td><td>96.33%</td><td>96.40%</td><td>97.8%</td><td>98.00%</td><td>100.0%</td></tr><tr><td></td><td>Nb FS</td><td>2</td><td>28.8</td><td>167.8</td><td>1741</td><td>10</td></tr><tr><td rowspan="2">Leukemia 3-C</td><td>Time</td><td>0.50</td><td>9.51</td><td>8.68</td><td>15.96</td><td>1.26</td></tr><tr><td>ACC</td><td>93.24%</td><td>90.96%</td><td>97.77%</td><td>94.57%</td><td>100.0%</td></tr><tr><td></td><td>Nb FS</td><td>3</td><td>46.6</td><td>349.2</td><td>1748</td><td>10</td></tr><tr><td rowspan="2">Leukemia 4-C</td><td>Time</td><td>0.50</td><td>9.32</td><td>12.92</td><td>13.98</td><td>0.94</td></tr><tr><td>ACC</td><td>94.54%</td><td>88.26%</td><td>93.63%</td><td>91.21%</td><td>100.0%</td></tr><tr><td></td><td>Nb FS</td><td>5</td><td>57.2</td><td>449.6</td><td>1771</td><td>8</td></tr></tbody></table>
<table><caption>Table 4 continued from previous page</caption><tr><td rowspan="3">Breastcancer</td><td>ACC</td><td>0.66</td><td>10.41</td><td>2.67</td><td>13.79</td><td>1.44</td></tr><tr><td>Time</td><td>96.58%</td><td>65.5%</td><td>76.19%</td><td>82.81%</td><td>98.57%</td></tr><tr><td>Nb FS</td><td>5</td><td>61.6</td><td>75.8</td><td>9255</td><td>15</td></tr><tr><td rowspan="3">Lung cancer</td><td>Time</td><td>0.98</td><td>14.13</td><td>62.08</td><td>15.46</td><td>3.86</td></tr><tr><td>ACC</td><td>95.17%</td><td>91.46%</td><td>92.58%</td><td>95.17%</td><td>100.0%</td></tr><tr><td>Nb FS</td><td>8</td><td>101.6</td><td>665.6</td><td>3437</td><td>20</td></tr><tr><td rowspan="3">Ovarian</td><td>Time</td><td>0.59</td><td>10.90</td><td>4.05</td><td>62.09</td><td>1.58</td></tr><tr><td>ACC</td><td>98.81%</td><td>97.53%</td><td>100.0%</td><td>100.0%</td><td>100.0%</td></tr><tr><td>Nb FS</td><td>3</td><td>71.8</td><td>42.6</td><td>5003</td><td>3</td></tr><tr><td rowspan="3">Lymphoma</td><td>Time</td><td>0.49</td><td>7.51</td><td>1.68</td><td>24.99</td><td>0.54</td></tr><tr><td>ACC</td><td>95.03%</td><td>96.66%</td><td>97.5%</td><td>100.0%</td><td>100.0%</td></tr><tr><td>Nb FS</td><td>2</td><td>25.4</td><td>41.2</td><td>1574</td><td>2</td></tr><tr><td rowspan="3">MLL</td><td>Time</td><td>0.97</td><td>8.81</td><td>8.92</td><td>20.56</td><td>0.59</td></tr><tr><td>ACC</td><td>95.99%</td><td>94.85%</td><td>98.92%</td><td>96.07%</td><td>100.0%</td></tr><tr><td>Nb FS</td><td>3</td><td>44.4</td><td>512.6</td><td>3303</td><td>4</td></tr></table>

## 4.4 Comparison with other works

In this section, we give a comparison of the results obtained by SVM-RFE and some other works presented in the literature. The results are presented in table 5. From table 5, we can see that the method proposed in [15] has achieved better classification accuracy with colon dataset and has selected less number of features. With leukemia, lung, ovarian, and lymphoma, both SVM-RFE and the method in [15] achieved 100% accuracy. With CNS and breast, SVM-RFE outperforms the method proposed in [15] in terms of classification accuracy and the number of selected features. SVM-RFE has better performance than the proposed method in [18] considering the only tested dataset (colon). SVM-RFE produced better performances than the method in [20] in terms of classification accuracy and the number of selected features for the ten datasets.

<table><caption>Table 5: Comparison with other works.</caption><thead><tr><th>Datasets</th><th></th><th>DT</th><th>RF</th><th>Lasso</th><th>Ridge</th><th>RFE-SVM</th></tr></thead><tbody><tr><td rowspan="2">Colon</td><td>Time</td><td>16.18</td><td>10.60</td><td>10.20</td><td>7.32</td><td>8414.06</td></tr><tr><td>ACC</td><td>75.46%</td><td>75.63%</td><td>85.76%</td><td>78.16%</td><td>75.16%</td></tr><tr><td rowspan="2">CNS</td><td>Time</td><td>71.03</td><td>13.86</td><td>49.57</td><td>8.82</td><td>181601.87</td></tr><tr><td>ACC</td><td>44.66%</td><td>62.83%</td><td>70.83%</td><td>72.5%</td><td>72.0%</td></tr><tr><td rowspan="2">Leukemia</td><td>Time</td><td>73.31</td><td>41.11</td><td>88.24</td><td>82.25</td><td>169159.77</td></tr><tr><td>ACC</td><td>87.44%</td><td>91.10%</td><td>97.71%</td><td>96.33%</td><td>93.14%</td></tr><tr><td rowspan="2">Leukemia 3-C</td><td>Time</td><td>85.84</td><td>39.79</td><td>168.67</td><td>77.07</td><td>159072.89</td></tr><tr><td>ACC</td><td>84.25%</td><td>84.62%</td><td>96.57%</td><td>94.57%</td><td>96.0%</td></tr><tr><td rowspan="2">Leukemia 4-C</td><td>Time</td><td>131.84</td><td>93.22</td><td>326.56</td><td>95.02</td><td>255629.21</td></tr><tr><td>ACC</td><td>69.90%</td><td>77.33%</td><td>92.92%</td><td>91.21%</td><td>94.64%</td></tr></tbody></table>
<table><caption>Table 5 continued from previous page</caption><tr><td rowspan="2">Breast</td><td>Time</td><td>443.38</td><td>34.45</td><td>182.76</td><td>35.85</td><td>4758377.28</td></tr><tr><td>ACC</td><td>52.49%</td><td>61.05%</td><td>72.43%</td><td>66.58%</td><td>64.44%</td></tr><tr><td rowspan="2">Lung cancer</td><td>Time</td><td>673.86</td><td>63.49</td><td>965.80</td><td>49.48</td><td>2016552.5</td></tr><tr><td>ACC</td><td>85.40%</td><td>88.55%</td><td>92.63%</td><td>93.28%</td><td>90.42</td></tr><tr><td rowspan="2">Ovarian</td><td>Time</td><td>639.34</td><td>225.18</td><td>352.96</td><td>435.79</td><td>2099057.88</td></tr><tr><td>ACC</td><td>96.27%</td><td>97.09%</td><td>100.0%</td><td>100.0%</td><td>100.0%</td></tr><tr><td rowspan="2">Lymphoma</td><td>Time</td><td>62.29</td><td>85.1%</td><td>50.37</td><td>79.70</td><td>37563.19</td></tr><tr><td>ACC</td><td></td><td></td><td>93.32%</td><td>94.16%</td><td>96.66%</td></tr><tr><td rowspan="2">MLL</td><td>Time</td><td>147.90</td><td>51.48</td><td>291.64</td><td>115.85</td><td>444714.36</td></tr><tr><td>ACC</td><td>80.54%</td><td>90.95%</td><td>98.57%</td><td>96.07%</td><td>90.23%</td></tr></table>

## 5 Conclusion and Future Work

In this paper we evaluate the performances of five embedded feature selection methods namely decision Tree, random forest, lasso, ridge, and SVM-RFE on ten cancer microarray gene expression datasets which are (Colon, CNS, Leukemia, Leukemia 3-C, Leukemia 4-C, breast, lung, ovarian, lymphoma, and MLL). The experiments show that SVM-RFE gives the highest accuracy among the five tested methods, using a reduced number of features. Decision tree provides high classification accuracy and selects the smallest number of features, it is the fastest method among the other methods discussed in this work and random forest produces the lowest classification accuracy. These are mathematical and algorithmic results, we believe that an expert opinion about the pertinence and the relevance of a specific feature in the final decision making is crucial especially in this domain because he is the only one who can ensure the stability criterion. This is the limitation of this study. As future work, we aim to consider more feature selection methods as well as use other evaluation metrics such as ROC. A main drawback of SVM-RFE is that the calculation time is very high. Therefore, we aim to develop a faster version of SVM-RFE in the future.

## References

1. Adorada, A., Permatasari, R., Wirawan, P.W., Wibowo, A., Sujiwo, A.: Support vector machine-recursive feature elimination (svm-rfe) for selection of microrna expression features of breast cancer. In: 2018 2nd International Conference on Informatics and Computational Sciences (ICICoS). pp. 1–4. IEEE (2018)

2. Bolón-Canedo, V., Alonso-Betanzos, A.: Microarray Bioinformatics. Springer (2019)

3. Breiman, L.: Bagging predictors. Machine learning 24(2), 123–140 (1996)

4. Díaz-Uriarte, R., De Andres, S.A.: Gene selection and classification of microarray data using random forest. BMC bioinformatics 7(1), 3 (2006)

5. Duval, B., Hao, J.K., Hernandez Hernandez, J.C.: A memetic algorithm for gene selection and molecular classification of cancer. In: Proceedings of the 11th Annual conference on Genetic and evolutionary computation. pp. 201–208 (2009)
6. Fonti, V., Belitser, E.: Feature selection using lasso. VU Amsterdam Research Paper in Business Analytics **30**, 1–25 (2017)

7. Grabczewski, K., Jankowski, N.: Feature selection with decision tree criterion. in null (2005)

8. Guyon, I., Weston, J., Barnhill, S., Vapnik, V.: Gene selection for cancer classification using support vector machines. Machine learning **46**(1-3), 389–422 (2002)

9. Hamed, T., Dara, R., Kremer, S.C.: An accurate, fast embedded feature selection for svms. In: 2014 13th International Conference on Machine Learning and Applications. pp. 135–140. IEEE (2014)

10. Hameed, S.S., Muhammad, F.F., Hassan, R., Saeed, F.: Gene selection and classification in microarray datasets using a hybrid approach of pcc-bpso/ga with multi classifiers. J. Comput. Sci. **14**(6), 868–880 (2018)

11. Hoerl, A.E., Kennard, R.W.: Ridge regression: Biased estimation for nonorthogonal problems. Technometrics **12**(1), 55–67 (1970)

12. Kumar, C.A., Sooraj, M., Ramakrishnan, S.: A comparative performance evaluation of supervised feature selection algorithms on microarray datasets. Procedia computer science **115**, 209–217 (2017)

13. Leung, M.K., Delong, A., Alipanahi, B., Frey, B.J.: Machine learning in genomic medicine: a review of computational problems and data sets. Proceedings of the IEEE **104**(1), 176–197 (2015)

14. Li, H., Guo, W., Wu, G., Li, Y.: A rf-pso based hybrid feature selection model in intrusion detection system. In: 2018 IEEE Third International Conference on Data Science in Cyberspace (DSC). pp. 795–802. IEEE (2018)

15. Li, Z., Xie, W., Liu, T.: Efficient feature selection and classification for microarray data. PloS one **13**(8), e0202167 (2018)

16. Liu, H., Zhou, M., Liu, Q.: An embedded feature selection method for imbalanced data classification. IEEE/CAA Journal of Automatica Sinica **6**(3), 703–715 (2019)

17. Ma, S., Song, X., Huang, J.: Supervised group lasso with applications to microarray data analysis. BMC bioinformatics **8**(1), 60 (2007)

18. Marafino, B.J., Boscardin, W.J., Dudley, R.A.: Efficient and sparse feature selection for biomedical text classification via the elastic net: Application to icu risk stratification from nursing notes. Journal of biomedical informatics **54**, 114–120 (2015)

19. Moon, M., Nakai, K.: Stable feature selection based on the ensemble l 1-norm support vector machine for biomarker discovery. BMC genomics **17**(13), 1026 (2016)

20. Remeseiro, B., Bolon-Canedo, V.: A review of feature selection methods in medical applications. Computers in biology and medicine **112**, 103375 (2019)

21. Tahir, N.M., Hussain, A., Samad, S.A., Ishak, K.A., Halim, R.A.: Feature selection for classification using decision tree. In: 2006 4th Student Conference on Research and Development. pp. 99–102. IEEE (2006)

22. Zhang, X., Shi, Z., Liu, X., Li, X.: A hybrid feature selection algorithm for classification unbalanced data processing. In: 2018 IEEE International Conference on Smart Internet of Things (SmartIoT). pp. 269–275. IEEE (2018)

23. Zheng, Y., Li, Y., Wang, G., Chen, Y., Xu, Q., Fan, J., Cui, X.: Retracted: A hybrid feature selection algorithm for microarray data. Concurrency and Computation: Practice and Experience **31**(12), e4716 (2019)

24. Zhu, Z., Ong, Y.S., Dash, M.: Markov blanket-embedded genetic algorithm for gene selection. Pattern Recognition **40**(11), 3236–3248 (2007)