# Improving Multi-Label Learning by Correlation Embedding
## 4 Experiment
### Evaluation Metrics  
A great many evaluation metrics have been proposed to evaluate the performance of multi-label learning algorithms. In the paper, we choose six common evaluation metrics. Define a test data \(\mathcal{T}=\{(\mathbf{x}_{1},Y_{1}),(\mathbf{x}_{2},Y_{2})\ldots(\mathbf{x}_{1}, Y_{n_{t}})\}\), where the ground truth labels set of the instance \(x_{i}\) is represented as \(Y_{i}\in\{0,1\}^{q}\), \(Y_{i}\in\mathcal{Y}\), \(h(\mathbf{x}_{i})\in\{0,1\}^{q}\) is the set of predicted class labels for the \(i\)-th instance, \(f(\mathbf{x}_{i},y)\) is the the confidence score that \(\mathbf{x}_{i}\) belongs to label \(y\). **Hamming Loss** evaluates the error between the predicted label of each instance obtained by the model and the true label of each instance. \[\text{Hamming Loss}=\frac{1}{n_{t}}\sum\limits_{i=1}^{n_{t}}\frac{1}{l}|h( \mathbf{x}_{i})\Delta Y_{i}| \tag{13}\]  
where \(\Delta\) indicates the symmetric difference between two sets. **One Error** evaluates the proportion of instances whose top-ranked label is not in the ground truth label set. \[\text{One Error}=\frac{1}{n_{t}}\sum\limits_{i=1}^{n_{t}}[[\arg\max\limits_{y \in\mathcal{Y}}f(\mathbf{x}_{i},y)]\in Y_{i}] \tag{14}\]  
where \([\cdot]\) represents the indication function. **Ranking Loss** indicates how many irrelevant labels are ranked higher than related labels. \[\text{Ranking Loss}=\frac{1}{n_{t}}\sum\limits_{i=1}^{n_{t}}\frac{1}{|Y_{i}| |\dot{Y}_{i}|}|\{(y^{\prime},y^{\prime\prime})|f(\mathbf{x}_{i},y^{\prime}) \leq f(\mathbf{x}_{i},y^{\prime\prime}),(y^{\prime},y^{\prime\prime})\in Y_{i }\times\bar{Y}_{i}\}| \tag{15}\]  
**Average Precision** evaluates the proportion of the label that is ranked before the relevant label of the instance is still the related label. \[\text{Average Precision}=\frac{1}{n_{t}}\sum\limits_{i=1}^{n_{t}}\frac{1}{|Y_{i }|}\sum\limits_{y\in Y_{i}}\frac{|\{y^{\prime}|rank_{f(\mathbf{x}_{i},y^{ \prime})}\leq rank_{f(\mathbf{x}_{i},y)},y^{\prime}\in Y_{i}\}|}{rank_{f( \mathbf{x}_{i},y)}} \tag{16}\]  
**Micro F1-Measure** evaluates the prediction performance of the learned classifier on the label set. \[\text{MicroF1}=\frac{2\sum\limits_{i=1}^{n_{t}}|h(\mathbf{x}_{i})\cap Y_{i}| }{\sum\limits_{i=1}^{n_{t}}|Y_{i}|+\sum\limits_{i=1}^{n_{t}}|h(\mathbf{x}_{i })|} \tag{17}\]  
**Example-based F1** is the integrated version of precision and recall for each instance. \[\text{Example-based F1}=\frac{1}{n_{t}}\sum\limits_{i=1}^{n_{t}}\frac{2p_{i }r_{i}}{p_{i}+r_{i}} \tag{18}\]  
where \(p_{i}\) and \(r_{i}\) are the precision and recall for the \(i\)-th instance. **Macro AUC** evaluates the probability that a positive instance is ranked before a negative instance, averaged over all labels. \[\text{AUC}=\frac{1}{l}\sum\limits_{i=1}^{l}\frac{|\{(\mathbf{x}^{\prime}, \mathbf{x}^{\prime\prime})|f(\mathbf{x}^{\prime},y_{j})\geq f(\mathbf{x}^{ \prime\prime},y_{j}),(\mathbf{x}^{\prime},\mathbf{x}^{\prime\prime})\in Z_{j} \times\bar{Z}_{j}\}|}{|Z_{j}||Z_{j}|} \tag{19}\]  
where \(Z_{j}=\{\mathbf{x}_{i}|y_{j}\in Y_{i},1\leq i\leq l\}(\bar{Z}_{j}=\{\mathbf{x }_{i}|y_{j}\
otin Y_{i},1\leq i\leq l\})\) indicates that it does not belong to a set of test instances labeled \(y_{j}\). For the AUC and AP evaluation metrics, the larger the value, the better the classification result. Hamming loss, One Error,Ranking Loss, and Coverage value are smaller, indicating better classification performance.