# Coverage-Based Classification Using Association Rule Mining +
## 6 Experimental Evaluation  
We tested our model on 12 real-life datasets taken from the UCI Machine Learning Repository. We evaluated our classifier (J&B) by comparing it with 8 well-known classification methods on the accuracy, relevance measures and the number of classification rules. All differences were tested for statistical significance by performing a paired t-test (with a 95% significance threshold). J&B was run with default parameters _minimum support_ = 1% and _minimum confidence_ = 60% (on some datasets, however, _minimum support_ was lowered to 0.5% or even 0.1% to ensure enough CARs to be generated for each class value). For all other 8 rule learners, we used their WEKA workbench [56] implementation with default parameters. We applied 90% as a required coverage (training dataset) threshold that is the stopping point of selecting rules to our classifier described in Section 5.1. The description of the datasets and input parameters are shown in Table 5. \begin{table}
\begin{tabular}{c c c} \hline
**N** & **Classifier Rules** & **\# Classified Examples** \\ \hline
1 & \(\{\text{a1}=3,\text{a4}=1\}\rightarrow\{1\}\) & 3 \\
2 & \(\{\text{a2}=2,\text{a5}=5\}\rightarrow\{2\}\) & 1 \\
3 & \(\{\text{a1}=1,\text{a2}=5\}\rightarrow\{3\}\) & 2 \\
4 & \(\{\text{a1}=1,\text{a3}=5,\text{a4}=4\}\rightarrow\{3\}\) & 2 \\
5 & \(\{\text{a3}=1,\text{a5}=2\}\rightarrow\{1\}\) & 2 \\
6 & \(\{\text{a1}=1,\text{a3}=4,\text{a5}=2\}\rightarrow\{2\}\) & 2 \\ \hline \end{tabular}
\end{table} 
Table 4: Final classifier. \begin{table}
\begin{tabular}{c c c c c c} \hline
**Dataset** & **\# of Attributes** & **\# of Classes** & **\# of Records** & _Min Support_ & _Min Confidence_ \\ \hline Breast Can & 10 & 2 & 286 & 1\% & 60\% \\ Balance & 5 & 3 & 625 & 1\% & 50\% \\ Car.Evn & 7 & 4 & 1728 & 1\% & 50\% \\ Vote & 17 & 2 & 435 & 1\% & 60\% \\ Tic-Tac-Toe & 10 & 2 & 958 & 1\% & 60\% \\ Nursery & 9 & 5 & 12,960 & 0.5\% & 50\% \\ Hayes-root & 6 & 3 & 160 & 0.1\% & 50\% \\ Lymp & 19 & 4 & 148 & 1\% & 60\% \\ Spect.H & 23 & 2 & 267 & 0.5\% & 50\% \\ Adult & 15 & 2 & 45,221 & 0.5\% & 60\% \\ Chess & 37 & 2 & 3196 & 0.5\% & 60\% \\ Connect4 & 43 & 3 & 67,557 & 1\% & 60\% \\ \hline \end{tabular}
\end{table} 
Table 5: Description of datasets and AC algorithm parameters. Furthermore, all experimental results were produced by using a 10-fold cross-validation evaluation protocol. Experimental results on classification accuracies (average values over the 10-fold cross-validation with standard deviations) are shown in Table 6 (OC: Overall Coverage). Our observations (Table 6) on selected datasets show that our proposed classifier (J&B) achieved better performance than Decision Table (DT), C4.5, Decision Table and Naive Bayes (DTNB), Ripple Down Rules (RDR) and Simple Associative Classifier (SA) (84.9% and 79.3%, 83.6%, 81.5%, 83.6%, 82.3%) on average accuracy. Our proposed method achieved the best accuracy on \"Breast Cancer\", \"Hayes.R\" and \"Connect4\" datasets. Standard deviations ranged around 4.0 in all classification methods and were higher for all methods on \"Breast cancer\", \"Hayes.R\", \"Lymp\" and \"Connect4\" datasets (above 4); that is, the differences between accuracies fluctuated and were reasonably high in 10-fold cross-validation experiments. When the overall coverage is above 90%, our proposed method tends to get reasonably high accuracy on all datasets. On the other hand, overall coverage of \"J&B\" was lower than its accuracy on \"Vote\" and \"Nursery\" datasets. This fact is not surprising since uncovered examples get classified by the majority classifier. Statistically significant testing (wins/losses counts) on accuracy between J&B and other classification models is shown in Table 7. W: our approach was significantly better than compared algorithms, L: selected rule-learning algorithm significantly outperformed our algorithm, W-L: no significant difference has been detected in the comparison. Table 7 illustrates that the performance of the J&B method on accuracy was better than DTNB, DT, RDR and SA methods.