# Multilabel Text Classification with Label-Dependent Representation
## 2 Related Work
### Multilabel Text Classification  
Traditionally, the classification (or categorization) of texts has been defined as assigning a Boolean value (true or false) to each pair \(\big{(}\langle d_{j},c_{i}\rangle\big{)}\in D\times C\), where D is the domain of the documents (corpus) and \(C=\big{(}c_{1},\ldots,c_{|c|}\big{)}\) is the set of default labels (classes). If a document is categorized under only one label, that is, in some partition of the set of classes (non-overlapping categories) or under multiple labels at once (overlapping categories), it is called a 'one-label problem' or a'multilabel problem', respectively [20]. The most commonly studied case to solve text classification problems is that of 'one-label', and the main approach is the so-called Binary Classification (BC), where a document is classified either to the category \(c_{i}\) or its complement (\(-c_{i}\)). This approach can be extended and used to solve problems with more categories. Interest in the classification of multilabel texts has increased in recent years. Among the proposals presented by [21], the most commonly used approach is the so-called transformation of the problem. In a multilabel problem, there exists a finite number of labels \(L=\{\lambda_{j}:j=1\ldots l\}\), where \(\lambda_{j}\) corresponds to the \(j\)-th label, and to the set of documents labeled \(D=\{f\{\mathbf{x}_{i},\mathbf{Y}_{i}:i=1\ldots d\}\}\), where \(\mathbf{x}_{i}\) represents the feature vector and \(\mathbf{Y}_{i}\subset L\) is the set of labels for the \(i\)-th text. Table 1 presents an example of multilabel texts. It represents that Text 1 belongs to the classes 'Sports' and 'Politics', that Text 2 belongs to the classes 'Sciences' 'Politics', that Text 3 belongs only to the class 'Sports', and finally that Text 4 belongs to the classes 'Religion' and 'Sciences'. The approaches to solving this problem can be classified into two approaches: the transformation of the problem and the adaptation of the model [21]. The transformation approach to the problem is algorithm-independent, that is, it transforms the multilabel learning task into a single-label classification task. Consequently, this technique may be used via current methods. The most common problem transformation method, called Binary Relevance (BR), learns \(|L|\) binary classifiers \(H_{(\lambda_{j})}:X\rightarrow\{\lambda_{j},\
eg\lambda_{j}\}\), one for each different label \(\lambda_{j}\) in \(L\). Through the use of Binary Relevance, the original data set is transformed into \(|L|\) data sets \(D_{\lambda_{j}}\). Each \(D_{\lambda_{j}}\) labels each instance of text in \(D\) with \(\lambda_{j}\) if \(\lambda_{j}\) is contained in the instance, or \(\lambda_{j}\) if the text example does not contain the tag. BR provides the same solution for both single-label problems and multilabel problems using binary classifiers. For the classification of a new instance \(\mathbf{x}\), this method generates a set of labels as the union of the labels generated by the classifier \(|L|,H_{\text{BR}}(x)=\cup_{(\lambda_{j}\in L)}\)\(\{\lambda_{j}\}:H_{\lambda_{j}}(x)=\lambda_{j}\). This is usually the most common transformation, and it is the same solution used when trying to deal with a multi-class classification problem using binary classifiers (see Table 2). Another type of transformation of the problem is called a Label Powerset (LP). In this transformation, each set of labels is considered as a new category. Then, if we have LP combinations, it is possible to use \(|LP|\) binary classifiers, one for each new label. The data set is handled as a single-label type, and then a single-label classifier with multiple disjoint classes is built (see Table 3). \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Texts (\(d\))**Label (\(l\))** & **Sports** & **Religion** & **Sciences** & **Politics** \\ \hline Text 1 & x & & & x \\ Text 2 & & & x & x \\ Text 3 & x & & & \\ Text 4 & & x & x & \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Representation of a Set of multilabeled Examples.