# Simultaneous Instance and Attribute Selection for Noise Filtering
## 4 Discussion
\begin{table}
\begin{tabular}{l c c c c c c c c} \hline \hline \multirow{2}{*}{**Methods**} & \multicolumn{2}{c}{**0\% Noise**} & \multicolumn{2}{c}{**5\% Noise**} & \multicolumn{2}{c}{**10\% Noise**} & \multicolumn{2}{c}{**15\% Noise**} \\ \cline{2-9}  & **[PERSON]’s** & **[PERSON]’s** & **[PERSON]’s** & **[PERSON]’s** & **[PERSON]’s** & **[PERSON]’s** & **[PERSON]’s** & **[PERSON]’s** \\  & **Ranking** & _p-Value_ & **Ranking** & _p-Value_ & **Ranking** & _p-Value_ & **Ranking** & _p-Value_ \\ \hline IN-GA & 2.633 & - & 2.567 & - & 2.267 & - & 2.167 & - \\ KJ-GA & 3.100 & 0.641 & 2.767 & 0.841 & 2.800 & 0.594 & 2.867 & 0.484 \\ RMHC-FP1 & 3.400 & 0.443 & 3.667 & 0.271 & 3.833 & 0.117 & 3.533 & 0.172 \\ AKH-GA & 3.667 & 0.301 & 3.600 & 0.301 & 3.767 & 0.134 & 3.700 & 0.125 \\ ROFS & 3.733 & 0.271 & 4.167 & 0.110 & 4.100 & 0.067 & 4.133 & 0.049 \\ TCCS & 5.467 & _0.005_ & 5.567 & _0.003_ & 5.633 & _0.001_ & 6.100 & _0.000_ \\ SOFSA & 6.867 & _0.000_ & 6.600 & _0.000_ & 6.667 & _0.000_ & 6.800 & _0.000_ \\ DS & 7.333 & _0.000_ & 7.333 & _0.000_ & 7.200 & _0.000_ & 6.967 & _0.000_ \\ Baseline & 8.800 & _0.000_ & 8.733 & _0.000_ & 8.733 & _0.000_ & 8.733 & _0.000_ \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Friedman + Holm results for feature retention of 1-NN. Rejected hypotheses are in italics. or computing candidate feature sets. Unfortunately, these results were obtained at the cost of significantly increasing the classification error. Considering three performance measures leads us to a multi-objective optimization problem. For this task, we employ the Pareto optimality criterion to assess the overall performance of the compared algorithms. The Pareto frontier finds the optimal results in multi-objective problems. A result is Pareto optimal if there is no other result that improves it simultaneously in all objectives. In our case, the three objectives are minimizing the classifier error and the number of features and instances in the training data. Figure 7 presents the methods without significant differences in performance, according to the analyzed measures. The only algorithm in the Pareto frontier is ROFS because no other algorithm appears in the pool of best-performed methods for the three measures (i.e., it equates or surpasses all other compared algorithms). In addition, we also applied the Kolmogorov-Smirnov statistical test to compare the distributions of ROFS's averaged performance with respect to state-of-the-art methods. For this statistical analysis, we used the graphical interface available at [[https://www.statkingdom.com/kolmogorov-Smirnov-two-calculator.html](https://www.statkingdom.com/kolmogorov-Smirnov-two-calculator.html)]([https://www.statkingdom.com/kolmogorov-Smirnov-two-calculator.html](https://www.statkingdom.com/kolmogorov-Smirnov-two-calculator.html)) (accessed on 13 September 2024), including the computation's R code. Table 9 presents the results. Regarding the average performance distributions, the test found significant differences between the distributions of ROFS and all other methods in the Instance Retention and Feature Retention measures. It also found significant differences with IN-GA, KJ-GA, and RMHC-FP1 according to classifier error. \begin{table}
\begin{tabular}{l c c c} \hline \hline
**ROFS vs.** & **Classifier Error** & **Instance Retention** & **Feature Retention** \\ \hline AKH-GA & 0.2286 & _0.0286_ & _0.013_ \\ DS & 0.7714 & _0.0286_ & _0.013_ \\ IN-GA & _0.0286_ & _0.0286_ & _0.013_ \\ KJ-GA & _0.0286_ & _0.0286_ & _0.013_ \\ RMHC-FP1 & _0.0286_ & _0.0286_ & _0.013_ \\ SOFSA & 0.2286 & _0.0286_ & _0.013_ \\ TCCS & 0.2286 & _0.0286_ & _0.013_ \\ \hline \hline \end{tabular} The statistically significant results at 95\(\%\) confidence are highlighted in italics.  
\end{table} 
Table 9: \(p\)-value results of the Kolmogorov–Smirnov statistical test comparing the averaged performance of the algorithms. Figure 7: Comparison of preprocessing algorithms according to their overall performance regarding classifier error, instance selection, and feature selection measures.