# Multilabel Text Classification with Label-Dependent Representation
## 2 Related Work
### Multilabel Text Classification
The second method deals with adapting some specific learning models and algorithms so that they can handle the multilabeled data directly. These adaptations are achieved thanks to model adjustments, such as modifications to the classical formulations of statistics or information theory. The preprocessing of the documents to achieve a better representation can also be considered within this type of transformation. The classification of multilabel texts has also been approached by means of algorithms, which directly capture the characteristics of the multilabel problem. The authors of [22] propose a method based on fuzzy logic, where a multilabeled text can belong to one, or more than one category. The authors state that by incorporating fuzzy techniques, the method can overcome problems caused by high memory requirements or low performance. The authors of [23] focus on solving the limitations of the _Backpropagation Learning_ algorithm so that it can work with multilabeled data, and propose _Backpropagation Multilabel Learning (BP-MLL)_. In this proposal, a very simple neural network approach is used for large-scale multilabeled text classification tasks. The authors of [24] propose a model based on graph attention networks to capture the dependency structure between labels (MAGNET). The results of the proposed model are validated on five real-world MLTC data sets. The proposed model achieves a similar or better performance compared to the previous models of the latest generation. The MAGNET model framework uses attention layer graphs for classification to generate the inputs for a BiLSTM to generate the feature vectors that are encoded by the embeddings of the BERT [25] model. The input to the attention network graph is an adjacency matrix and the label vectors. The GAT output are label features that are applied to the feature vectors obtained with BiLSTM. It uses performance metrics that does not allow a person to evaluate its performance on unbalanced data sets. MAGNET's proposal improved performance marginally over traditional models, such as Binary Relevance and Chain Classifiers. In [26], the Label-Wise Pre-Training (LW-PT) method is suggested for obtaining a document representation that includes label information. It is expected that the linked labels always co-occur in related documents, and for this, a multilabel document is represented as a composite of various label representations. By creating document-label classifiers and instructing document-label coders, LW-PT puts this concept into practice. The previously trained label encoder fine-tunes. The experimental results support the suggested \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**TextsLabel** & **Sports** & **No Sports** & **TextsLabel** & **Religion** & **No Religion** \\ \hline Text 1 & X & Text 1 & X \\ Text 2 & X & Text 2 & X \\ Text 3 & X & Text 3 & X \\ Text 4 & & X & X \\ \hline
**TextsLabel** & **Politics** & **No Politics** & **TextsLabel** & **Sciences** & **No Sciences** \\ \hline Text 1 & X & Text 1 & X \\ Text 2 & X & Text 2 & X & \\ Text 3 & & X & Text 3 & X \\ Text 4 & & X & Text 4 & X \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Resulting Transformation using Binary Relevance. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Example** & **Label** & **Sports** & **Sports and Politics** & **Sciences and Politics** & **Sciences and Religion** \\ \hline Text 1 & & x & & \\ Text 2 & & & x & \\ Text 3 & x & & & \\ Text 4 & & & & x \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Resulting Transformation using Label Powerset. approach in two sets of data, and only a few evaluation measures are used to compare it to conventional approaches. On the other hand, ref. [27] proposes a global hierarchical model for the classification of multilabeled texts, which seeks to take advantage of the dependency relationships between labels. In line with the exploitation of Deep Learning, ref.