# Simultaneous Instance and Attribute Selection for Noise Filtering
## 4 Discussion
Despite the promising results obtained by ROFS and outlined before, to determine the best methods in each scenario, we applied the Friedman test followed by the Holm post hoc test at a 95% confidence level. This combination of non-parametric statistical tests is suggested in [31]. We used the KEEL software (v.3.0) [32] for the statistical analysis. In Supplementary Materials, under the folder \"statistics\", interested readers can find the full documentation reported by KEEL. In all cases, we select as null hypothesis H0 that there are no differences in the performance of the compared algorithms and as alternative hypothesis H1 that there are differences in the performance of the compared algorithms. In the comparisons, we include the classifier error with no preprocessing as a baseline. Table 6 presents the results of the statistical analysis for classifier error, considering the four scenarios. For each scenario, we include [PERSON]'s ranking as well as the probability values (\(p\)-values) obtained by [PERSON]'s post hoc tests comparing the first-ranked algorithm with respect to all others. However, we also want to minimize the number of features and instances in the training data. Tables 7 and 8 present the respective statistical analyses. In Tables 6-8, the compared methods are ordered according to the ranking obtained by [PERSON]'s test considering the noise-free scenario, and the cases in which the null hypothesis was rejected are highlighted in italics. For the noise-free scenario, the best result was obtained by the baseline 1-NN (the first in [PERSON]'s ranking). ROFS, SOFSA, AKH-GA, DS, and TCCS had classifier error results without significant differences. However, RMHC-FP1, KJ-GA, and IN-GA had significantly worse performance than the baseline 1-NN. For 5% of noise, ROFS had the best result, without significant differences with the baseline classifiers SOFSA, AKH-GA, and DS. These results point out that some evolutionary algorithms do not accurately represent the data, therefore increasing classifier error after preprocessing. The methods TCCS, RMHC-FP1, KJ-GA, and IN-GA had significantly worse performance than the proposed ROFS. As the noise increases to 10% and 15%, ROFS remains the best-performed method, without significant differences in classifier error with DS and the baseline classifier. All remaining preprocessing algorithms obtained significantly higher results for the classifier error measure. \begin{table}
\begin{tabular}{l c c c c c c c c} \hline \hline \multirow{2}{*}{**Methods**} & \multicolumn{3}{c}{**0\% Noise**} & \multicolumn{3}{c}{**5\% Noise**} & \multicolumn{3}{c}{**10\% Noise**} & \multicolumn{3}{c}{**15\% Noise**} \\ \cline{2-9}  & **[PERSON]’s** & **[PERSON]’s** & **[PERSON]’s** & **[PERSON]’s** & **[PERSON]’s** & **[PERSON]’s** & **[PERSON]’s** & **[PERSON]’s** \\  & **Ranking** & \(p\)**-Value** & **Ranking** & \(p\)**-Value** & **Ranking** & \(p\)**-Value** & **Ranking** & \(p\)**-Value** \\ \hline Baseline & 2.467 & - & 2.900 & 0.594 & 3.767 & 0.067 & 3.567 & 0.042 \\ ROFS & 3.633 & 0.243 & 2.367 & - & 1.933 & - & 1.533 & - \\ SOFSA & 4.167 & 0.089 & 4.633 & 0.023 & 4.900 & _0.003_ & 5.600 & _0.000_ \\ AKH-GA & 4.400 & 0.053 & 4.833 & 0.014 & 4.567 & _0.008_ & 5.033 & _0.000_ \\ DS & 4.433 & 0.049 & 3.167 & 0.424 & 2.667 & 0.463 & 2.900 & 0.172 \\ TCCS & 4.867 & 0.016 & 5.067 & _0.007_ & 5.500 & _0.000_ & 5.667 & _0.000_ \\ RMHC-FP1 & 6.333 & _0.000_ & 6.933 & _0.000_ & 6.233 & _0.000_ & 5.867 & _0.000_ \\ KJ-GA & 7.067 & _0.000_ & 7.900 & _0.000_ & 7.567 & _0.000_ & 7.133 & _0.000_ \\ IN-GA & 7.633 & _0.000_ & 7.200 & _0.000_ & 7.867 & _0.000_ & 7.700 & _0.000_ \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Friedman + Holm results for classifier error of 1-NN. Rejected hypotheses are in italics. The statistical analysis shows that noise significantly affects preprocessing algorithms but not ROFS and DS, therefore supporting the hypothesis that using an error-based editing approach contributes to maintaining classification performance.