# Comparison of Instance Selection and Construction Methods with Various Classifiers
## 4 Results and Analysis  
Since simple averaging has limited interpretability, we used both average performance and average rank to asses the quality of the evaluated methods. In order to make ranking for each dataset and each classifier the results obtained for particular data filters were ranked from the best to the worst in terms of classification accuracy and compression. The highest rank (equal to 26, which is the number of evaluated algorithms) was given to the best filter method for particular dataset and the lowest rank (1) was assigned to the worst method (rank with ties). Then the ranks over datasets were averaged to indicate the final performance. Such a comparison does not reflect how much one method differs from the other in terms of given quality measure, but ranking unlike averaging performances is insensitive to the data distribution where measures like accuracy can range from 40% on one dataset up to 99% on another. On the other hand ranking do not provide information on how much the methods differ so these both quality measures complement each other and should be considered together, where the ranking gives an answer which method was more often better, and then, the mean performance indicates how much given method was better from the competitor. Moreover, the threshold obtained by the random sampling should be applied simultaneously to both values and when any of them is below the threshold given method should be rejected as useless. The obtained results including both average ranks as well as average performances are presented in Table 3. Moreover, the Wilcoxon signed-rank statistical test [36] was used to check whether the results obtained by the classifier without any data filter significantly differ from the results obtained when given data filter was used to cleanup the dataset. The calculations were conducted with significance level \(\alpha=0.1\). The data filters which did not lead to a significantly decrease of the prediction accuracy were marked with =. In few cases the data filter allowed to increase the accuracy of a classifier, and if the increase was statistically significant we marked the results with + sign. In this case the significance was measured using Wilcoxon tailed sign-rank test. \begin{table}
\begin{tabular}{c l l l l} \hline \hline
**Id.** & **Name** & **s / a / c** & **Id.** & **Name** & **s / a / c** \\ \hline
1 & appendicitisis & 106 / 7 / 2 & 21 & page-blocks & 5472 / 10 / 5 \\
2 & balance & 625 / 4 / 3 & 22 & phoneme & 5404 / 5 / 2 \\
3 & banana & 5300 / 2 / 2 & 23 & pima & 768 / 8 / 2 \\
4 & bands & 365 / 19 / 2 & 24 & ring & 7400 / 20 / 2 \\
5 & bupa & 345 / 6 / 2 & 25 & satimage & 6435 / 36 / 6 \\
6 & Cleveland & 297 / 13 / 5 & 26 & segment & 2310 / 19 / 7 \\
7 & glass & 214 / 9 / 6 & 27 & sonar & 208 / 60 / 2 \\
8 & haberman & 306 / 3 / 2 & 28 & spambase & 4597 / 57 / 2 \\
9 & hayes-roth & 160 / 4 / 3 & 29 & spectfheart & 267 / 44 / 2 \\
10 & heart & 270 / 13 / 2 & 30 & tae & 151 / 5 / 3 \\
11 & hepatitis & 80 / 19 / 2 & 31 & texture & 5500 / 40 / 11 \\
12 & ionosphere & 351 / 33 / 2 & 32 & thyroid & 7200 / 21 / 3 \\
13 & iris & 150 / 4 / 3 & 33 & titanic & 2201 / 3 / 2 \\
14 & led7 digit & 500 / 7 / 10 & 34 & twonorm & 7400 / 20 / 2 \\
15 & mammographic & 830 / 5 / 2 & 35 & vehicle & 846 / 18 / 4 \\
16 & marketing & 6876 / 13 / 9 & 36 & vowel & 990 / 13 / 11 \\
17 & monk-2 & 432 / 6 / 2 & 37 & wdbc & 569 / 30 / 2 \\
18 & movement\_libras & 360 / 90 / 15 & 38 & wine & 178 / 13 / 3 \\
19 & newthyroid & 215 / 5 / 3 & 39 & wisconsin & 683 / 9 / 2 \\
20 & optdigits & 5620 / 64 / 10 & 40 & yeast & 1484 / 8 / 10 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Datasets used in the experiments. The s/a/c denotes the number of samples, attributes and classes. To increase readability, the results which represent ranks are also presented graphically independently for each classifier.