# Outlier Detection Based Feature Selection Exploiting Bio-Inspired Optimization Algorithms
## 7 Experimentation Results  
The proposed outlier detection-based FS techniques (PPFS) were tested using a Dell XPS 9343 with an Intel core TM i7-5500U CPU 2.40 GHz and 8 GB RAM. In the following experimentation, 19 datasets (described in Table 2) were used to validate the effectiveness of this study. The first nine datasets are from the UCI repository ([[https://archive.ics.uci.edu/ml/datasets.php](https://archive.ics.uci.edu/ml/datasets.php)]([https://archive.ics.uci.edu/ml/datasets.php](https://archive.ics.uci.edu/ml/datasets.php)) (accessed on 14 January 2021)). The next eight datasets are from ([[https://www.opemml.org/search?q=autralian&type=data](https://www.opemml.org/search?q=autralian&type=data)]([https://www.opemml.org/search?q=autralian&type=data](https://www.opemml.org/search?q=autralian&type=data)) (accessed on 14 January 2021)). Finally, August_senate_polls (August S.P) dataset is taken from [[https://data.world/fivethirtyeight/august-senate-polls](https://data.world/fivethirtyeight/august-senate-polls)]([https://data.world/fivethirtyeight/august-senate-polls](https://data.world/fivethirtyeight/august-senate-polls)) (accessed on 14 January 2021), and Fifa2019 is from [[https://www.kaggle.com/karangadiya/fifa19](https://www.kaggle.com/karangadiya/fifa19)]([https://www.kaggle.com/karangadiya/fifa19](https://www.kaggle.com/karangadiya/fifa19)) (accessed on 14 January 2021). August S.P contains pols from 1990 to 2016, the results of the election related to each poll, and the average error of each poll. This dataset is composed of three classes. The Fifa2019 dataset is composed of more than 13,000 instances and 80 attributes. An extensive preprocessing operation was applied to this dataset. Then, two classes were created in the body_type attribute, namely Lean and Normal, using R programming language. As EPP-Lab software is used for outlier detection, only the number of iterations and the number of runs are set. All the parameters related to GA (set in EPP-Lab [38]) were not changed. This is why no validation phase for setting the parameters of GA is needed. In fact, for GA, the population is randomly generated with 50 individuals. The tournament selection was used with three individuals. The crossover was employed with two points, and the probability equals 0.65. The mutation is applied to all individuals with a probability 0.05. As stated before, the Tribes method does not require parameters. For the initialization phase, the EPP Lab randomly generates an initial solution (population) for each method. The numbers of iterations and runs are set to 100 according to the recommendations of [38]. Figure 5: Projection Pursuit-based feature-selection approach. The number of runs guarantees that the obtained results (the removed outliers) cannot be improved. It also allows determining the number of times a feature is detected as an outlier. The threshold that confirms the outlying status is then set to 50 (see Section 6.1). For more details, one can refer to [38; 44]. In the classification stage, three classifiers are applied: Support Vector Machine (SVM) with a Linear kernel, the k Nearest Neighbors (k-NN), and the Random Forest (RF). R (version 3.5.1) is used for the classification step. The datasets are split into training (70%) and testing (30%). The split was done randomly by R using the \"createDataPartition\" function from the caret library. To validate the parameter setting of the classifiers, ten-fold Cross-Validation, repeated ten times, is applied to the training set. The accuracy with the largest value is used to select the optimal training model. The optimal model with its best parameters' values is then used in the testing stage. The training and validation accuracy values are omitted to save space. The training set is centered and scaled. For the k-NN classifier, several values of k were tested (between 5 and 200) for each dataset. For the RF classifier, the number of attributes used for splitting at each node in the tree was studied.