# Online Streaming Feature Selection via Conditional Independence
## 5 Experiments and Analysis
### Comparison of ConInd with Two Online Algorithms
#### 5.3.1 Prediction Accuracy  
As illustrated in Figure 7, we summarize the prediction accuracy for the 12 classifiers on the 14 datasets during online learning. The labels of the x-axis from 1 to 14 denote the datasets: (1) wdbc; (2) colon; (3) lucas0; (4) sylva; (5) ionosphere; (6) cina0; (7) lucap0; (8) marti1; (9) reged1; (10) Lung; (11) prosate_GE; (12) leukemia; (13) arcene; and (14) Smk_can_187.  
We conducted experiments on these datasets using \(G^{2}\) tests for discrete data and _Fisher's Z-test_ for continuous data at a significance level \(\alpha=0.05\). The prediction accuracies of _ConInd_ and _OSFS_ were higher than that of _Alpha-investing_ on 5-14 datasets in these classifiers. _ConInd_ consistently achieved higher accuracy in all classifiers except for RUSBoosted Trees. As shown in Figure 7, the accuracies of the classifiers were overtly reduced in the leukemia, marti1, and reged1 datasets. As shown in the three curves in Figure 7, we observed that _Alpha-investing_, _OSFS_, and _ConInd_ have the same prediction accuracies in some datasets. There are seven datasets in ComplexTree, two in Medium Tree, four in SimpleTree, five in Liner SVM, four in Quadratic SVM, four in Cubic SVM, five in FineKNN, four in Medium KNN, four in Cubic KNN, four in Bagged Trees, four in Subspace discriminant, and three in RUSBoosted Trees. Prediction accuracies for the wdbc, lucas0, and sylva datasets were equal, except for RUSBoosted Trees, because they have the same respective selected features.  
Table 5 presents the results of average accuracy in three different algorithms for the 14 datasets. For the average of Decision Tree, SVM, KNN, and ENSEMBLE, _ConInd_ offers higher average accuracy (i.e., 88.79, 88.63, 9.76, and 88.61, respectively) than _Alpha-investing_ (i.e., 83.74, 84.08, 83.2, and 82.3, respectively) does. The average classification accuracy of the features selected using the _ConInd_ algorithm is the highest among the three algorithms. It is important to note that the average accuracy of _ConInd_ is 5.62% higher than that of _Alpha-investing_.  
Figure 7: Prediction accuracy of algorithms in the 14 datasets under different classifiers.  
\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline \multirow{2}{*}{**Algorithms**} & \multicolumn{4}{c}{**Average Accuracy for Classifiers in 14 the Datasets (\%)**} & \multicolumn{2}{c}{**Average Accuracy (\%)**} \\ \hline \multirow{6}{*}{_Alpha-investing_} & Complex & Medium & Simple & Linear & Quadratic & \multirow{2}{*}{Cubic SVM} & \multirow{2}{*}{\multirow{2}{*}{83.33}} \\  & Tree & Tree & Tree & SVM & SVM & & \\ \cline{1-1} \cline{5-7}  & 83.89 & 84.14 & 83.18 & 86.34 & 85.13 & 80.78 & \\ \cline{1-1} \cline{5-7}  & Decision & Tree average: & 83.74 & & SVM average: & 84.08 & \\ \cline{1-1} \cline{2-7}  & Fine & Medium & Cubic & Bagged & Subspace & RUSBoosted \\ \cline{1-1} \cline{2-7}  & KNN & KNN & KNN & Trees & discriminant & Trees \\ \cline{1-1} \cline{2-7}  & 82.31 & 83.75 & 83.54 & 84.91 & 86.19 & 75.81 & \\ \cline{1-1} \cline{2-7}  & \multicolumn{4}{c}{KNN average: 83.2} & \multicolumn{4}{c}{ENSEMBLE average: 82.3} \\ \hline \hline \end{tabular}
\end{table}
Table 5: Comparison of average prediction accuracies.  
In our experiments, we found that _ConInd_ has better accuracy than _Alpha-investing_. As mentioned above, this is because _Alpha-investing_ only evaluates each feature once instead of considering the redundancy of selected features. As a result, the prediction accuracy is low and unstable. Similarly, _ConInd_ also shows slightly better average accuracy than _OSFS_. A possible explanation is that a few parts of strongly relevant features and non-redundant features may be discarded due to the characteristics of streaming features.