# Ensemble of Networks for Multilabel Classification
## 3 DataSets
### Performance Indicators  
In the experimental section, several performance indicators are used to evaluate the classifiers on the multiclass benchmarks.  
Let X be a dataset with m samples \(\mathbf{x}_{i}\in\Re^{d}\). A given sample has an actual label \(\mathbf{y}_{i}\in\{0,1\}^{l}\), where l is the number of total labels. Let H and F be the set of predicted labels, where \(\mathbf{h}_{i}\in\{0,1\}^{l}\) is the predicted label vector for sample \(\mathbf{x}_{i}\), and \(\mathbf{f}_{i}\in\Re^{l}\) is the confidence relevance of each prediction. The performance indicators are defined for H and F as follows:  
* Hamming loss is the fraction of misclassified labels, \[\text{HLoss}(H)= \frac{1}{ml}\sum_{i=1}^{m}\sum_{j=1}^{l}I(\mathbf{y}_{i}(j)\
eq\mathbf{h}_ {i}(j))\,\] (1) where \(I(0)\) is the indicator function. Hamming loss must be minimized. When hamming loss is 0, there is no error in the predicted label vector.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Name** & **\#Patterns** & **\#Features** & **\#Labels** & **LCard** \\ \hline CAL500 & 502 & 68 & 174 & 26.044 \\ Image & 2000 & 294 & 5 & 1.236 \\ Scene & 2407 & 294 & 5 & 1.074 \\ Yeast & 2417 & 103 & 14 & 4.24 \\ Arts & 5000 & 462 & 26 & 1.636 \\ ATC & 3883 & 42 & 14 & 1.265 \\ ATC\_f & 3883 & 700 & 14 & 1.265 \\ Liu & 832 & 2892 & 1385 & 71.160 \\ mAn & 3916 & 20 & 20 & 1.650 \\ bibtex & 7395 & 1836 & 159 & 2.402 \\ enron & 1702 & 1001 & 53 & 3.378 \\ health & 5000 & 612 & 32 & 1.662 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Datasets summary.  
One error is the fraction of instances whose most confident label is incorrect. The indicator should be minimized: \[\text{OneError}(\text{F})=\frac{1}{m}\sum_{i=1}^{m}I\left(\mathbf{h}_{i}\left(\underset {j}{\text{argmax}}\mathbf{f}_{i}\right)\ \
eq\ \mathbf{y}_{i}\left(\underset{j}{\text{argmax}}\mathbf{f}_{i}\right)\ \right)\,\] (2)
* Ranking loss is the average fraction of reversely ordered label pairs for each instance. It is derived from the confidence value by taking into account the number of confidence values correctly ranked (i.e., when a true label is ranked before a wrong label). Ranking loss is also an error. Therefore, it should be minimized.
* Coverage is the average number of steps needed to move down the ranked label list of an instance to cover all its relevant labels. As such, coverage should be minimized.
* Average precision is the average fraction of relevant labels ranked higher than a particular label. As such, average precision should be maximized. Another set of indicators adopted by many researchers [60]:include:
* Aiming is the ratio of correctly predicted labels and practically predicted labels: \[\text{Aiming}(H)= \frac{1}{m}\sum_{i=1}^{m}\frac{||\mathbf{h}_{i}\cap\mathbf{y}_{i}||}{||\bm {h}_{i}||}\] (3)
* Recall is the rate of the correctly predicted labels and actual labels: \[\text{Recall}(H)= \frac{1}{m}\sum_{i=1}^{m}\frac{||\mathbf{h}_{i}\cap\mathbf{y}_{i}||}{||\bm {y}_{i}||}\.\] (4)
* Accuracy is the average ratio of correctly predicted labels over total labels: \[\text{Accuracy}\ (H)= \frac{1}{m}\sum_{i=1}^{m}\frac{||\mathbf{h}_{i}\cap\mathbf{y}_{i}||}{||\bm {h}\cup\mathbf{y}_{i}||}\.\] (5)
* Absolute true is the ratio of the perfectly correct prediction events and the total number of prediction events: \[\text{AbsTrue}(H)= \frac{1}{m}\sum_{i=1}^{m}I(\mathbf{h}_{i}=\mathbf{y}_{i})\.\] (6)
* Absolute false is the ratio of the completely wrong prediction events and total number of prediction events: \[\text{AbsFalse}(H)= \frac{1}{m}\sum_{i=1}^{m}\frac{||\mathbf{h}_{i}\cup\mathbf{y}_{i}||-|| \mathbf{h}_{i}\cap\mathbf{y}_{i}||}{l}\.\] (7)  
All ten indicators lie within the range [0, 1] and should be maximized, except for Absolute false.