Hierarchical Multi-label Text Classification with Horizontal and Vertical Category Correlations (2021)

Abstract: Hierarchical multi-label text classification (HMTC) deals with the challenging task where an instance can be assigned to multiple hierarchically structured categories at the same time. The majority of prior studies either focus on reducing the HMTC task into a flat multi-label problem ignoring the vertical category correlations or exploiting the dependencies across different hierarchical levels without considering the horizontal correlations among categories at the same level, which inevitably leads to fundamental information loss. In this paper, we propose a novel HMTC framework that considers both vertical and horizontal category correlations. Specifically, we first design a loosely coupled graph convolutional neural network as the representation extractor to obtain representations for words, documents, and, more importantly, level-wise representations for categories, which are not considered in previous works. Then, the learned category representations are adopted to capture the vertical dependencies among levels of category hierarchy and model the horizontal correlations. Finally, based on the document embeddings and category embeddings, we design a hybrid algorithm to predict the categories of the entire hierarchical structure. Extensive experiments conducted on real-world HMTC datasets validate the effectiveness of the proposed framework with significant improvements over the baselines.

## introduction as a fundamental problem in natural language processing (nlp), text classification is the task of assigning a given document to one or multiple categories according to its textual content. in practice, many documents are tagged with multiple categories that can be organized in a tree or a directed acyclic graph (dag) @xcite , which poses a more challenging task. these categories can be organized into different levels of the hierarchical structure, and the task of assigning multiple hierarchically structured categories to documents is known as hierarchical multi-label text classification (hmtc). for the hierarchical multi-label classification problem, it is essential to model the dependencies among categories in the hierarchical structure. the vertical correlations capture the dependencies of categories at different levels, and the horizontal correlations reflect the relationships at the same level. straightforwardly, the hmtc problem can be reduced to a flat multi-label problem @xcite , which simply ignores the vertical category correlations. to address that, attempts have been made to exploit the hierarchical dependencies to improve the classification performance. among them, @xcite adopts a top-down way to generate a hierarchical structure of decision-treebased classifiers to predict categories at the corresponding hierarchical level. @xcite proposes a unified framework that combines the local outputs of each category hierarchical level and the global output of the entire network. @xcite designs a hierarchical attention-based memory unit to model the dependencies among different levels in a top-down fashion. however, horizontal correlations between categories at the same hierarchical level are usually ignored, resulting in a lack of information transfer of label characteristics at the same level. in principle, the characteristics of categories are encoded in both the horizontal correlations among categories at the same level and the vertical dependencies between categories at different levels in the hierarchical organization. precisely, the vertical correlation measures the top-down relevance of a text node's tags, while the horizontal correlation is responsible for enhancing the information transfer of labels within the same hierarchical level. an example is shown in figure 1 . the red area reveals the vertical dependency of categories, which has been widely exploited in hmtc tasks. in the meantime, the blue area represents the horizontal correlation of categories, which is latent but essential. for instance, when we are confident that the document is tagged with hierarchical classification, it is more likely that the document is associated with multi-label classification rather than single label classification. to fully leverage the hierarchical structure of categories, this paper integrates the horizontal and vertical correlations extracted by a novel graph convolutional neural network (gcn). the graph convolutional neural network @xcite has been successfully applied to text classification due to its capabilities in handling complex structures. a gcn-based text classification model generally works by treating the training documents, test documents, and words as nodes to construct a single huge graph, which incurs the following issues in hierarchical text classification. firstly, during network training, the category information which provides valuable semantic information is missing. secondly, organizing all nodes in one graph expands the size of the graph, which not only leads to the increased difficulty of training, but also brings information confusion between nodes. to address the problems discussed above, in this paper, we propose an integrated framework named horizontally and vertically hierarchical multi-label text classification (hvhmc) to exploit the vertical and horizontal category correlations simultaneously by introducing a newly designed loosely coupled graph convolutional network (lcgcn) as the representation component. specifically, we include the category nodes together with word nodes and document nodes in lcgcn by constructing two separate graphs: the core graph containing words and categories, and the document-word graph. then, based on the category representations learned based on the loosely coupled graph neural network, a level-wise category correlation matrix is calculated, which captures the horizontal dependencies among categories and facilitates the semantic transfer among categories at each hierarchical level. finally, a hybrid algorithm is proposed to further incorporate vertical dependencies of categories by integrating the label information on the hierarchical path. the predictions of each hierarchical level and the overall hierarchical structure will be combined as the final results. the contributions of our paper are as follows: • we propose the loosely coupled graph convolutional neural network (lcgcn) as the representation component. by dealing with the core graph and the document-word graph separately, this feature extraction approach can greatly enhance the quality of semantic representations and reduce the training expenses while maintaining the performance; ## related works in this section, we mainly review the related studies on hierarchical multi-label classification and graph convolutional neural networks. ## methodology this section introduces the proposed framework of horizontally and vertically hierarchical multilabel text classification (hvhmc) in details. problem definition in the hmtc problem, there are a set of documents, each document contains the text description and its expected categories, which are organized in a hierarchical structure. before defining the hmtc problem, we first give a detailed description of the hierarchical structure and documents. given the possible categories in h hierarchical levels @xmath0, c 2 , • • • , c h ), where ## experiments in this section, we conduct extensive experiments on three real-world datasets to evaluate our proposed approach. we first introduce the datasets and baselines followed by the experimental results. ## conclusion this paper proposes a novel hierarchical multilabel text classification approach named horizontally and vertically hierarchical multi-label text classification (hvhmc). we first design a loosely coupled graph convolutional neural network as the representation layer, capturing the word-to-word, category-to-word, and category-to-category associations. after the category representations are learned, both the horizontal and vertical category correlations are considered to facilitate the hierarchical classification process. finally, extensive experiments are conducted to verify the effectiveness of the proposed framework.