Label-Specific Dual Graph Neural Network for Multi-Label Text Classification (2021)

Abstract: Multi-label text classification is one of the fundamental tasks in natural language processing. Previous studies have difficulties to distinguish similar labels well because they learn the same document representations for different labels, that is they do not explicitly extract label-specific semantic components from documents. Moreover, they do not fully explore the high-order interactions among these semantic components, which is very helpful to predict tail labels. In this paper, we propose a novel label-specific dual graph neural network (LDGN), which incorporates category information to learn label-specific components from documents, and employs dual Graph Convolution Network (GCN) to model complete and adaptive interactions among these components based on the statistical label co-occurrence and dynamic reconstruction graph in a joint way. Experimental results on three benchmark datasets demonstrate that LDGN significantly outperforms the state-of-the-art models, and also achieves better performance with respect to tail labels.

## introduction automatically labeling multiple labels of documents is a fundamental and practical task in natural language processing. recently, with the growth of data scale, multi-label text classification(mltc) has attracted more attention, since it is often applied to many fields such as sentiment analysis @xcite , emotion recognition @xcite , web page tagging @xcite and so on. however, the number of labels and documents and the complex relations of labels render it an unsolved and challenging task. existing studies for multi-label text classification mainly focus on learning enhanced document *corresponding author representation @xcite and modeling label dependency @xcite @xcite to improve classification performance. although they have explored the informative words in text content, or considered the label structure and label semantics to capture label correlations, these models cannot distinguish similar labels well (e.g., the categories prices vs consumer prices in reuters news). the main reason is that most of them neglect the semantic connections between labels and input documents and they learn the same document representations for different labels, which cannot issue the label similarity problem. more specifically, they do not explicitly consider the corresponding semantic parts of each label in the document. recently, some studies @xcite @xcite have used attention mechanism to explore the above semantic connections, and learn a label-specific document representation for classification. these methods have obtained promising results in mltc, which shows the importance of exploring semantic connections. however, they did not further study the interactions between label-specific semantic components which can be guided by label correlations, and thus these models cannot work well on predicting tail labels which is also a challenging issue in mltc. to handle these issues, a common way to explore the semantic interactions between labelspecific parts in document is to utilize the statistical correlations between categories to build a label co-occurrence graph for guiding interactions. nevertheless, statistical correlations have three drawbacks. first, the co-occurrence patterns between label pairs obtained from training data are incomplete and noisy. specifically, the label cooccurrences that appear in the test set but do not appear in the training set may be ignored, while some rare label co-occurrences in the statistical correlations may be noise. second, the label cooccurrence graph is built in global, which may be biased for rare label correlations. and thus they are not flexible to every sample document. third, statistical label correlations may form a long-tail distribution, i.e., some categories are very common while most categories have few of documents. this phenomenon may lead to models failing to predict low-frequency labels. thus, our goal is to find a way to explore the complete and adaptive interactions among label-specific semantic components more accurately. in this paper, we investigate: (1) how to explicitly extract the semantic components related to the corresponding labels from each document; and (2) how to accurately capture the more complete and more adaptive interactions between label-specific semantic components according to label dependencies. to solve the first challenge, we exploit the attention mechanism to extract the labelspecific semantic components from the text content, which can alleviate the label similar problem. to capture the more accurate high-order interactions between these semantic components, we first employ one graph convolution network (gcn) to learn component representations using the statistical label co-occurrence to guide the information propagation among nodes (components) in gcn. then, we use the component representations to reconstruct the adjacency graph dynamically and re-learn the component representations with another gcn, and thus we can capture the latent interactions between these semantic components. finally, we exploit final component representations to predict labels. we evaluate our model on three real-world datasets, and the results show that the proposed model ldgn outperforms all the comparison methods. further studies demonstrate our ability to effectively alleviate the tail labels problem, and accurately capture the meaningful interactions between label-specific semantic components. the contributions of this paper are as follows: ## model as depicted in figure 1 , our model ldgn is composed of two major modules: 1) labelspecific document representation 2) dual graph neural network for semantic interaction learning. specifically, label-specific document representation learning describes how to extract labelspecific semantic components from the mixture of label information in each document; and the dual graph neural network for semantic interaction learning illustrates how to accurately explore the complete interactions among these semantic components under the guidance of the prior knowledge of statistical label co-occurrence and the posterior information of dynamic reconstruction graph. problem formulation: let @xmath0, y i } n be the set of documents, which consists of n document x i and its corresponding label y @xmath1, 1} |c| , where |c| denotes the total number of labels. each document x i contains j words x @xmath2, w i2 , . . . , w ij . the target of multi-label text classification is to learn the mapping from input text sequence to the most relevant labels. ## conclusion in this paper, we propose a graph-based network ldgn to capture the semantic interactions related to corresponding labels, which jointly exploits global statistical patterns and local dynamic relations to derive complete and adaptive dependencies between different label-specific semantic parts. we first exploit multi-label attention to extract the label-specific semantic components from documents. then, we employ gcn to learn component representations using label co-occurrences to guide the information propagation among components. after that, we use the learned component representations to compute the adjacency graph dynamically and re-learn with gcn based on the reconstruction graph. extensive experiments con-ducted on three public datasets show that the proposed ldgn model outperforms other state-ofthe-art models on multi-label text classification task and also demonstrates much higher effectiveness to alleviate the tail label problem. in the future, we will improve the proposed model in efficiency, for example we could construct a dynamic graph for a few samples rather than each sample. and besides, we will explore more information about labels for mlc classification.