# Multilabel Text Classification with Label-Dependent Representation
## 4 Experimental Method
### Performance Measures  
Traditional evaluation measures such as the \(F\) measure, Hamming Loss, and Accuracy are useful in the case of multilabeled sets.  
To describe the performance measures, the following notation was used: considering the vector \(\mathbf{Y}_{i}\in[0,1]^{|L|}:i=1\ldots d\), then each label will be relevant if \(y_{i,j}=1\), and for its part, the prediction of the classifier will be \(y^{\prime}_{i,j}=1\), where \(d\) is the number of documents and \(|L|\) is the number of possible labels.  
Based on the notation above, Hamming Loss is defined as in Equation (6):  
\[Hamming\text{-}Loss(Y,Y^{\prime})=\frac{1}{d}\frac{1}{|L|}\sum_{i=1}^{d}\sum_{j= 1}^{|L|}\Bigl{|}y^{\prime}_{i,j}\Delta y_{i,j}\Bigr{|} \tag{6}\]  
where \(y^{\prime}_{i,j}\Delta y_{i,j}\) represents the difference between the labels assigned by the classifier and the actual labels. This measure seeks to measure the difference between each label that the texts actually have, with each label that the classifier assigned to said texts. The lower the value obtained, the better the performance.  
Another multilabel measure is the label set precision (Accuracy), and this is defined as in Equation (7):  
\[Accuracy(y,y^{\prime})=\frac{1}{d}\sum_{i=1}^{d}1(y_{i}=y^{\prime}_{i}). \tag{7}\]  
This average performance allowed us to measure, for each text, the correctly assigned labels. In multilabel classification, the function returns the precision of the subset. If the entire set of predicted labels for a sample strictly matches the actual set of labels, then the subset precision is 1; otherwise, it is 0. The higher the returned value, the better the performance.  
The \(F\) measure, commonly used in information retrieval, is very popular in multilabeled text classification. The measure \(F\) is the harmonic mean between precision and completeness (_recall_). The measure \(F\) (\(F_{1}\)) for each label was calculated as shown in Equation (8):  
\[F_{1}(Y_{i},Y^{\prime}_{i})=\frac{2\times\textit{precision}\times\textit{ recall}}{\textit{precision}+\textit{recall}}, \tag{8}\]  
where Accuracy is the fraction of the predictions that are actually relevant, and recall is the fraction of actual relevance with respect to the predictions. The higher the value of \(F\), the better the performance.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Data Set** & **Num of Labels** & **Num of Docs** & **Size of Voc** & **Card** & **Dens** & **Div** \\ \hline Enron & 52 & 1.702 & 1.001 & 3.378 & 0.064 & 0.442 \\ Bibtex & 159 & 7.395 & 1.836 & 2.402 & 0.015 & 0.386 \\ TMC2007 & 22 & 28.600 & 49.060 & 2.158 & 0.098 & 0.047 \\ Ohsumed & 23 & 13.929 & 1.002 & 1.663 & 0.072 & 0.082 \\ Education & 33 & 12.030 & 27.530 & 1.463 & 0.044 & 0.042 \\ Reuters & 103 & 6.000 & 500 & 1.462 & 0.014 & 0.135 \\ Science & 40 & 6.428 & 37.490 & 1.450 & 0.036 & 0.071 \\ Medical & 45 & 978 & 1.449 & 1.245 & 0.028 & 0.096 \\ Slashdot & 22 & 3.782 & 1.079 & 1.181 & 0.054 & 0.041 \\ Langlog & 75 & 1.460 & 1.004 & 1.180 & 0.016 & 0.208 \\ \hline \hline \end{tabular}
\end{table}
Table 5: Characteristics of the Preprocessed Data Set. Cardinality (Card) measures the average number of labels associated with each document. Density (Dens) is defined as the cardinality divided by the number of labels. The Diversity (Div) represents the percentage of label sets present in the set divided by the number of possible label sets. Vocabulary Size considers the volume of distinct words.  
For the multilabel case, it is necessary to combine the different \(F_{1}\) of each evaluation of the label. For that we use macro-\(F_{1}\), which is the average of \(F_{1}\) for each label.