# Intelligent Neural Network Schemes for Multi-Class Classification
## 4 Experimental Results
### Effects of Dimensionality Reduction  
We apply several dimensionality reduction techniques to avoid overfitting and improve performance for RBF networks. Different techniques may have different effects. One technique is good for some datasets but is not good for other datasets. Unfortunately, there are no universal guidelines about the selection of dimensionality reduction techniques for a given dataset. Usually, trial and error is necessary. Table 5 shows the testing ACC values obtained by SL-Scheme with different dimensionality reduction techniques for some single-label datasets, while Table 6 shows the testing HL values obtained by ML-Scheme with different dimensionality reduction techniques for some multi-label datasets. Note that WO indicates no dimensionality reduction is applied, and MI and IG mean mutual information and information gain, respectively. Dimensionality reduction is not always good. For example, for the yeast-sl dataset for single-label classification and the genbase dataset for multi-label classification, our proposed schemes perform best without any dimensionality reduction. As can be seen from the above two tables, no dimensionality reduction technique is the best for all the datasets. Pearson does not have good effects for single-label datasets, but it has good effects for multi-label datasets. On the other hand, MI has good effects for single-label datasets, but it does not so for multi-label datasets. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Dataset** & **ML-SVM** & **DT** & **ML-KNN** & **MLP** & **LVQ** & **RandForest** & **ML-Scheme** \\ \hline birds & 0.1154 & 0.2237 & 0.0510 & 0.2496 & 0.2259 & 0.2291 & **0.0460** \\ scene & 0.1568 & 0.2094 & 0.1130 & 0.2933 & 0.2325 & 0.1994 & **0.0602** \\ emotions & 0.2343 & 0.2428 & 0.2302 & 0.2967 & 0.1652 & 0.2192 & **0.1497** \\ yeast-ml & 0.2108 & 0.2787 & 0.2136 & 0.3122 & **0.1059** & 0.3130 & 0.1429 \\ cal500 & 0.1380 & 0.2137 & 0.1400 & 0.1679 & 0.2286 & 0.1567 & **0.1263** \\ genbase & **0.0015** & 0.0302 & 0.0020 & 0.1366 & 0.1765 & 0.1194 & 0.0016 \\ flags & 0.2791 & 0.213 & 0.3099 & 0.2636 & 0.1493 & 0.2497 & **0.1466** \\ corel5k & 0.0090 & 0.0720 & 0.0654 & 0.0968 & 0.1160 & 0.0406 & **0.0088** \\ mediamill & 0.0335 & 0.1158 & 0.1024 & 0.1567 & 0.1752 & 0.1267 & **0.0204** \\ \hline average & 0.1309 & 0.1775 & 0.1364 & 0.2193 & 0.1750 & 0.1838 & **0.0781** \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Testing Hamming Loss (HL) obtained by different methods. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Dataset** & **\# Attribute** & **\# Instances** & **\# Categories** & **Cardinality** \\ \hline birds & 260 & 645 & 19 & 1.014 \\ scene & 294 & 2407 & 6 & 1.074 \\ emotions & 72 & 593 & 6 & 1.869 \\ yeast-ml & 103 & 2417 & 14 & 4.237 \\ cal500 & 68 & 502 & 174 & 26.044 \\ genbase & 1186 & 662 & 27 & 1.252 \\ flags & 19 & 194 & 7 & 3.392 \\ corel5k & 499 & 5000 & 374 & 3.522 \\ mediamill & 120 & 43,907 & 101 & 4.376 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Characteristics of the datasets for multi-label multi-class classification. In addition, multiple dimensionality reduction techniques can be used simultaneously to improve performance of RBF networks. Figure 6 shows the performance of SL-Scheme with MI, LDA, and MI+LDA, respectively, for some single-label datasets. As can be seen, MI performs best for the libras and drivface datasets, LDA performs best for the heart and breast datasets, while MI+LDA performs best for the pd-speech and balance-scale datasets. Figure 7 shows the performance of ML-Scheme with Pearson, LDA, and Pearson+LDA, respectively, for some multi-label datasets. As can be seen, Pearson performs best for the yeast-ml, genbase, and mediamill datasets, while Pearson+LDA performs best for the cal500, flags, and corel5K datasets.