# Intelligent Neural Network Schemes for Multi-Class Classification
## 4 Experimental Results
### Multi-Label Multi-Class Classification  
Next, we show the performance of different methods on multi-label multi-class classification. The metric \"Hamming Loss\" (HL) is used for performance evaluation, defined as [43]  
\[\text{Hamming Loss(HL)}=\frac{\sum_{i=1}^{N_{t}}H_{d}(\mathbf{y}_{i},\mathbf{o} _{i})}{mN_{t}} \tag{27}\]  
where \(N_{t}\) is the number of testing instances and \(H_{d}(\mathbf{a},\mathbf{b})\) is the Hamming distance between \(\mathbf{a}\) and \(\mathbf{b}\). Note that, instead of counting the number of correctly classified instances like ACC, HL uses Hamming distance to calculate the mismatch between the original string of target categories and the string of predicted categories for every testing instance and then calculates the average across the dataset. For multi-label classification, HL is a more suitable metric than ACC. Ideally, we would expect HL = 0, which implies no error, for perfect classification. Practically, the smaller the value of HL, the better the classification performance.  
Nine multi-label benchmark datasets, taken from the MULAN library [46], are used in this experiment. The characteristics of these datasets are shown in Table 3. Each dataset contains collected instances for multi-label classification in a different situation. For example, the birds dataset is a benchmark for ecological investigations of birds; the cal500 dataset is a popular dataset used in music autotagging, described as coming from \"500 songs\"; The flags dataset contains details of various nations and their flags, etc. Note that an instance in a multi-label dataset may belong to more than one category. The Cardinality column indicates the number of categories on average an instance belongs to. The cardinality of a dataset can be greater than 1. For example, the cardinality of the yeast-ml dataset is 4.237, indicating each instance belongs to 4.237 categories in average.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Dataset** & **SVM** & **DT** & **KNN** & **MLP** & **LVQ** & **BandForest** & **SL-Scheme** \\ \hline Iris & 0.9660 & 0.9600 & 0.9689 & 0.9822 & **0.9933** & 0.9511 & 0.9788 \\ soybean & **1** & **1** & 0.9894 & 0.9400 & **1** & 0.9389 & 0.9889 \\ glass & 0.9070 & 0.9065 & 0.9081 & 0.9423 & 0.6733 & **0.9922** & 0.9783 \\ yeast-sl & 0.9185 & 0.9098 & 0.9197 & 0.9055 & 0.3187 & 0.9156 & **0.9362** \\ ecoli & 0.8720 & 0.9509 & 0.9635 & 0.8366 & 0.8930 & 0.8285 & **0.9661** \\ car & 0.9818 & 0.9777 & **0.9864** & 0.9737 & 0.9770 & 0.9792 & 0.9656 \\ madelon & 0.6746 & 0.7223 & 0.6646 & 0.5035 & 0.6639 & 0.7227 & **0.7546** \\ wine & 0.9888 & 0.9476 & 0.9813 & 0.5694 & 0.7868 & 0.9517 & **0.9924** \\ sonar & 0.8413 & 0.7548 & 0.8029 & 0.8271 & 0.8799 & 0.7264 & **0.9031** \\ libras & 0.9804 & 0.9507 & **0.9811** & 0.9596 & 0.7028 & 0.9719 & 0.9641 \\ heart & 0.8407 & 0.7704 & 0.8333 & 0.7556 & **0.8704** & 0.7481 & 0.8593 \\ breast & 0.9772 & 0.9051 & 0.9631 & 0.5182 & 0.9261 & 0.9157 & **0.9789** \\ drivface & 0.9692 & 0.9560 & 0.9692 & 0.9582 & 0.9587 & 0.9648 & **1.0000** \\ pd-speech & 0.8532 & 0.7870 & 0.9061 & 0.7434 & 0.8577 & 0.8095 & **0.9987** \\ balance-scale & 0.9445 & 0.8507 & 0.8592 & **0.9776** & 0.9267 & 0.8645 & 0.9413 \\ \hline average & 0.9413 & 0.8900 & 0.9131 & 0.8262 & 0.8288 & 0.8854 & **0.9453** \\ \hline \hline \end{tabular}
\end{table}
Table 2: Testing accuracy (ACC) obtained by different methods.  
Table 4 shows the testing HL values obtained by different methods. In this table, we compare our method, ML-Scheme, with six other methods. ML-SVM is a multi-label version of SVM, and ML-KNN is a multi-label version of KNN. The codes of ML-SVM and ML-KNN are taken from the website [[https://scikit-learn.org/](https://scikit-learn.org/)]([https://scikit-learn.org/](https://scikit-learn.org/)). From this table, we can see that SL-Scheme performs best in Hamming loss for seven out of nine datasets. As can be seen, ML-Scheme performs best, having the lowest average HL 0.0781.